diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 39b38497f..e3c35bb4d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -70,12 +70,12 @@ Using Nvidia as an example: ```bash pip install .[dev] \ - -C cmake.define.INFINI_RT_ROOT=/path/to/infinirt-prefix \ + -C cmake.define.INFINI_RT_ROOT=/path/to/infini-rt-prefix \ -C cmake.define.WITH_CPU=ON \ -C cmake.define.WITH_NVIDIA=ON ``` -`/path/to/infinirt-prefix` is the InfiniRT install prefix, typically the value +`/path/to/infini-rt-prefix` is the InfiniRT install prefix, typically the value used for InfiniRT's `CMAKE_INSTALL_PREFIX`. It should contain `include/infini/rt.h` and `lib/libinfinirt.so`. @@ -83,7 +83,7 @@ Auto-detection is supported for some platforms, so you can also let InfiniOps detect the device backends while still pointing it at the installed InfiniRT: ```bash -pip install .[dev] -C cmake.define.INFINI_RT_ROOT=/path/to/infinirt-prefix +pip install .[dev] -C cmake.define.INFINI_RT_ROOT=/path/to/infini-rt-prefix ``` > `[dev]` installs optional development dependencies (e.g. `pytest`) that are not needed for production but required for development and testing. After the first install, subsequent installs only need `pip install .`. @@ -102,7 +102,7 @@ For routine development and pull requests, start with a smoke build plus the smo ```bash python -m pip install .[dev] --no-build-isolation --no-deps \ - --config-settings=cmake.define.INFINI_RT_ROOT=/path/to/infinirt-prefix \ + --config-settings=cmake.define.INFINI_RT_ROOT=/path/to/infini-rt-prefix \ --config-settings=cmake.define.INFINI_OPS_SMOKE_BUILD=ON python -m pytest tests -m smoke -q ``` diff --git a/README.md b/README.md index ba648dcea..1a00710c3 100644 --- a/README.md +++ b/README.md @@ -15,10 +15,10 @@ InfiniOps is a high-performance, cross-platform operator library supporting mult Install InfiniRT first, then build InfiniOps with the InfiniRT install prefix: ```bash -pip install . -C cmake.define.INFINI_RT_ROOT=/path/to/infinirt-prefix +pip install . -C cmake.define.INFINI_RT_ROOT=/path/to/infini-rt-prefix ``` -`/path/to/infinirt-prefix` is the directory passed to InfiniRT as +`/path/to/infini-rt-prefix` is the directory passed to InfiniRT as `CMAKE_INSTALL_PREFIX`; it should contain `include/infini/rt.h` and `lib/libinfinirt.so`. @@ -27,7 +27,7 @@ platforms explicitly: ```bash pip install . \ - -C cmake.define.INFINI_RT_ROOT=/path/to/infinirt-prefix \ + -C cmake.define.INFINI_RT_ROOT=/path/to/infini-rt-prefix \ -C cmake.define.WITH_CPU=ON \ -C cmake.define.WITH_NVIDIA=ON ``` @@ -59,9 +59,18 @@ For Hygon builds, set `DTK_ROOT` to the DTK installation root if it is not insta See [CONTRIBUTING.md](CONTRIBUTING.md) for code style, commit conventions, PR workflow, development guide, and troubleshooting. -## Development Docs +## Documentation +Full user and contributor documentation starts at [docs/README.md](docs/README.md). + +Useful entry points: + +- [Getting Started](docs/getting-started.md) +- [Build and Test](docs/build.md) +- [Backends](docs/backends.md) +- [Operators](docs/api/operators.md) - [Adding ATen-backed operators](docs/aten-operators.md) +- [Compatibility](docs/compatibility.md) ## License diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 000000000..9951212cb --- /dev/null +++ b/docs/README.md @@ -0,0 +1,37 @@ +# InfiniOps Documentation + +InfiniOps is a high-performance operator library for InfiniCore. It provides +Python bindings and C++ operator implementations across CPU, NVIDIA, Iluvatar, +Hygon, MetaX, Cambricon, Moore, Ascend, and other backends. + +The common public include entry is: + +```cpp +#include +``` + +The Python package entry is: + +```python +import infini.ops +``` + +Start with these pages: + +- [Getting Started](getting-started.md): install InfiniRT, build InfiniOps, and + run a minimal operator call. +- [Build and Test](build.md): build options, smoke builds, and test commands. +- [Backends](backends.md): supported backend options and backend-specific + requirements. +- [Core Types](api/core-types.md): tensors, devices, data types, handles, and + configuration objects. +- [Operators](api/operators.md): the operator API model, dispatch, caching, and + implementation layout. +- [Adding ATen-backed operators](aten-operators.md): generated and hand-written + PyTorch ATen backend guidance. +- [Compatibility](compatibility.md): public API boundary, generated files, and + internal implementation headers. + +Generated API reference pages are intentionally left out of this first +documentation pass. They should be added through a dedicated Doxygen build and +publishing change. diff --git a/docs/api/core-types.md b/docs/api/core-types.md new file mode 100644 index 000000000..ab24946cc --- /dev/null +++ b/docs/api/core-types.md @@ -0,0 +1,78 @@ +# Core Types + +This page summarizes the core types used by InfiniOps operators. InfiniOps +reuses the public InfiniRT runtime types where possible. + +## Tensor + +`infini::ops::Tensor` is an alias for `infini::rt::TensorView`. + +```cpp +#include "tensor.h" + +std::vector values(16); +infini::ops::Tensor tensor{ + values.data(), + std::vector{4, 4}, + infini::ops::Device{infini::ops::Device::Type::kCpu, 0}}; +``` + +`Tensor` is a non-owning view. It stores the data pointer, shape, data type, +device, and strides, but it does not own the memory it references. + +## Device + +`infini::ops::Device` is an alias for `infini::rt::Device`. + +Known device types include: + +- `Device::Type::kCpu` +- `Device::Type::kNvidia` +- `Device::Type::kIluvatar` +- `Device::Type::kMetax` +- `Device::Type::kMoore` +- `Device::Type::kHygon` +- `Device::Type::kCambricon` +- `Device::Type::kAscend` + +Some source code also carries experimental or future device enum values. Treat +documented build options in [Backends](../backends.md) as the supported user +surface. + +## DataType + +`infini::ops::DataType` is imported from InfiniRT. Common values include: + +- `DataType::kInt8`, `DataType::kInt16`, `DataType::kInt32`, `DataType::kInt64` +- `DataType::kUInt8`, `DataType::kUInt16`, `DataType::kUInt32`, `DataType::kUInt64` +- `DataType::kFloat16`, `DataType::kBFloat16`, `DataType::kFloat32`, `DataType::kFloat64` + +InfiniOps also exposes type-list helpers such as `FloatTypes`, +`ReducedFloatTypes`, `IntTypes`, `UIntTypes`, and `AllTypes` for template +dispatch. + +## Handle + +`Handle` carries optional runtime resources for a call: + +- backend stream +- workspace pointer +- workspace size + +Operators that need temporary memory read the workspace from the handle. +Callers that do not need custom stream or workspace handling can use the +shorter `Op::Call(...)` form without an explicit handle. + +## Config + +`Config` carries operator configuration that is independent of tensor geometry. +The most common field is `implementation_index`, which selects one of the +active implementations for an operator and device. + +```cpp +infini::ops::Config config; +config.set_implementation_index(1); +``` + +Use the default config unless a specific backend implementation needs to be +selected deliberately. diff --git a/docs/api/operators.md b/docs/api/operators.md new file mode 100644 index 000000000..4ecf5c7c4 --- /dev/null +++ b/docs/api/operators.md @@ -0,0 +1,99 @@ +# Operators + +InfiniOps operators are C++ classes with generated Python bindings. They share +a common dispatch model across devices and backend implementations. + +## Public Call Shape + +Python users call generated functions from `infini.ops`: + +```python +import infini.ops + +infini.ops.gemm(a, b, out) +``` + +C++ development code calls operator classes: + +```cpp +infini::ops::Gemm::Call(a, b, out); +``` + +For calls that need a stream, workspace, or implementation selection, pass +`Handle` and `Config` explicitly: + +```cpp +infini::ops::Handle handle; +infini::ops::Config config; + +config.set_implementation_index(1); +infini::ops::Gemm::Call(handle, config, a, b, out); +``` + +## Dispatch Model + +Each operator has: + +- a base class under `src/base/.h` +- zero or more native implementations under `src/native/.../ops//` +- optional PyTorch C++ implementations under `src/torch/ops//` +- generated wrappers and Python bindings under `generated/` +- tests under `tests/test_.py` + +`Operator` specializations provide concrete +implementations. `Device` selects the backend and `Index` selects the +implementation slot for that operator on that backend. + +## Implementation Indexes + +Implementation indexes are local to an operator and device. Use an explicit +index only when the backend exposes multiple implementations for the same +operator. + +Generated ATen-backed wrappers reserve implementation index `8`. Hand-written +backend implementations must avoid colliding with existing implementations for +the same operator. + +## Operator Cache + +`Operator::Call(...)` caches constructed operator instances per thread. The +cache key includes the config implementation index and tensor/scalar geometry. +Tests can call `clear_cache()` on generated Python operator classes for module +isolation. + +Code that changes tensor geometry, backend implementation selection, or +workspace assumptions should account for this caching behavior. + +## Adding an Operator + +The standard path for a native operator is: + +1. Add the base class in `src/base/.h`. +2. Add one or more backend implementations under `src/native/.../ops//`. +3. Add or update generated wrapper inputs if needed. +4. Add focused tests under `tests/test_.py`. +5. Validate a smoke build plus the focused test on every affected backend. + +For PyTorch ATen-backed operators, see +[Adding ATen-backed operators](../aten-operators.md). That page explains the +generated backend path and the hand-written ATen backend path. + +## Smoke Coverage + +Smoke builds use an operator allowlist to keep routine validation short: + +```bash +python -m pip install .[dev] --no-build-isolation --no-deps \ + --config-settings=cmake.define.INFINI_RT_ROOT=/path/to/infini-rt-prefix \ + --config-settings=cmake.define.WITH_CPU=ON \ + --config-settings=cmake.define.INFINI_OPS_SMOKE_BUILD=ON +``` + +Then run: + +```bash +python -m pytest tests -m smoke -q +``` + +Use full builds and broader tests for shared dispatch, wrapper generation, +backend infrastructure, or high-risk operator changes. diff --git a/docs/backends.md b/docs/backends.md new file mode 100644 index 000000000..de33a93b6 --- /dev/null +++ b/docs/backends.md @@ -0,0 +1,72 @@ +# Backends + +InfiniOps can be built for CPU and one accelerator backend at a time. The +Python and operator APIs remain common, while device SDKs, compiler flags, and +available implementations differ by backend. + +## Backend Options + +| Backend | CMake option | Notes | +| --- | --- | --- | +| CPU | `WITH_CPU` | Used as the smallest build and can be enabled with one accelerator backend. | +| NVIDIA | `WITH_NVIDIA` | Requires CUDA Toolkit. | +| Iluvatar | `WITH_ILUVATAR` | CUDA-compatible backend using the CoreX toolchain. | +| Hygon | `WITH_HYGON` | Requires DTK. `DTK_ROOT` defaults to `/opt/dtk` when unset. | +| MetaX | `WITH_METAX` | Requires the MetaX runtime and SDK paths. | +| Cambricon | `WITH_CAMBRICON` | Requires Cambricon Neuware. | +| Moore | `WITH_MOORE` | Requires MUSA Toolkit through `MUSA_ROOT`, `MUSA_HOME`, `MUSA_PATH`, or `/usr/local/musa`. | +| Ascend | `WITH_ASCEND` | Requires Ascend CANN and, by default, the custom AscendC kernel toolchain. | +| PyTorch C++ | `WITH_TORCH` | Adds ATen-backed implementations when PyTorch C++ headers and libraries are available. | + +## Device Auto-Detection + +`AUTO_DETECT_DEVICES=ON` probes device files such as `/dev/nvidia*` and turns on +matching backend options. This is useful on configured developer machines but +can be too implicit for reproducible CI or release builds. + +Prefer explicit backend options in scripts, CI, and release instructions. + +## Backend Selection in Tests + +The Python test harness accepts platform names through `--devices`, for example: + +```bash +python -m pytest tests -m smoke -q --devices cpu nvidia +``` + +Supported selector names include: + +- `nvidia` +- `metax` +- `iluvatar` +- `hygon` +- `moore` +- `cambricon` +- `ascend` + +The harness maps those platform names to the PyTorch device type used by the +installed backend, such as `cuda`, `musa`, `mlu`, or `npu`. + +## Implementation Layout + +Backend implementations live under: + +```text +src/native///ops// +``` + +Examples include: + +- `src/native/cpu/ops/gemm/` +- `src/native/cuda/nvidia/ops/gemm/` +- `src/native/ascend/ops/matmul/` +- `src/native/cambricon/ops/rms_norm/` + +The PyTorch C++ backend uses: + +```text +src/torch/ops// +generated/torch// +``` + +Generated files are build artifacts and should not be edited by hand. diff --git a/docs/build.md b/docs/build.md new file mode 100644 index 000000000..20c457eb6 --- /dev/null +++ b/docs/build.md @@ -0,0 +1,105 @@ +# Build and Test + +InfiniOps uses CMake, scikit-build-core, and Python packaging. The most common +entry is `python -m pip install` with CMake options passed through +`--config-settings`. + +## Common CMake Options + +| Option | Description | Default | +| --- | --- | :---: | +| `WITH_CPU` | Enable the CPU backend. | `OFF` | +| `WITH_NVIDIA` | Enable the NVIDIA CUDA backend. | `OFF` | +| `WITH_ILUVATAR` | Enable the Iluvatar CUDA-compatible backend. | `OFF` | +| `WITH_HYGON` | Enable the Hygon backend. | `OFF` | +| `WITH_METAX` | Enable the MetaX backend. | `OFF` | +| `WITH_CAMBRICON` | Enable the Cambricon backend. | `OFF` | +| `WITH_MOORE` | Enable the Moore backend. | `OFF` | +| `WITH_ASCEND` | Enable the Ascend backend. | `OFF` | +| `WITH_TORCH` | Enable PyTorch C++ ATen-backed operators. | `OFF` | +| `WITH_NINETOOTHED` | Enable NineToothed-generated kernels. | `OFF` | +| `AUTO_DETECT_DEVICES` | Auto-detect available device files. | `OFF` | +| `AUTO_DETECT_BACKENDS` | Auto-detect available backend packages. | `OFF` | +| `GENERATE_OPERATOR_CALL_INSTANTIATIONS` | Generate explicit C++ operator call instantiations. | `ON` | +| `GENERATE_PYTHON_BINDINGS` | Generate Python bindings. | `OFF` in raw CMake, `ON` in `pyproject.toml` | +| `INFINI_RT_ROOT` | InfiniRT install prefix containing `include/` and `lib/`. | `$INFINI_RT_ROOT` | +| `INFINI_OPS_SMOKE_BUILD` | Build only the smoke-test operator subset. | `OFF` | +| `INFINI_OPS_OPS` | Comma- or semicolon-separated operator allowlist. | empty | +| `INFINI_OPS_TORCH_OPS` | Comma- or semicolon-separated ATen operator allowlist. | empty | + +Only one GPU backend should be enabled in a build. CPU may be enabled with the +selected accelerator backend. + +## Python Wheel Build + +Using CPU as the smallest backend: + +```bash +python -m pip install .[dev] \ + --config-settings=cmake.define.INFINI_RT_ROOT=/path/to/infini-rt-prefix \ + --config-settings=cmake.define.WITH_CPU=ON +``` + +Using NVIDIA as an example accelerator backend: + +```bash +python -m pip install .[dev] \ + --config-settings=cmake.define.INFINI_RT_ROOT=/path/to/infini-rt-prefix \ + --config-settings=cmake.define.WITH_CPU=ON \ + --config-settings=cmake.define.WITH_NVIDIA=ON +``` + +The built wheel installs the InfiniOps Python extension and the InfiniRT shared +library needed by the extension. + +## Smoke Build + +For routine development and pull requests, start with a smoke build: + +```bash +python -m pip install .[dev] --no-build-isolation --no-deps \ + --config-settings=cmake.define.INFINI_RT_ROOT=/path/to/infini-rt-prefix \ + --config-settings=cmake.define.WITH_CPU=ON \ + --config-settings=cmake.define.INFINI_OPS_SMOKE_BUILD=ON +``` + +`INFINI_OPS_SMOKE_BUILD=ON` narrows generated wrappers, bindings, and generated +Torch ops to a representative operator subset. Use full builds for release +preparation, shared build or dispatch changes, wrapper generation changes, and +platform maintainer spot checks. + +## Tests + +Run the full test suite: + +```bash +python -m pytest +``` + +Run the smoke set: + +```bash +python -m pytest tests -m smoke -q +``` + +Select platforms explicitly: + +```bash +python -m pytest tests -m smoke -q --devices cpu nvidia +``` + +The platform names accepted by the test harness include `nvidia`, `metax`, +`iluvatar`, `hygon`, `moore`, `cambricon`, and `ascend`. The harness maps those +names to the corresponding PyTorch device type when needed. + +## Formatting + +Run the checks that match the touched files: + +```bash +ruff format --check . +ruff check . +``` + +C++ changes should also pass the repository `clang-format` and `clang-tidy` +expectations described in `CONTRIBUTING.md`. diff --git a/docs/compatibility.md b/docs/compatibility.md new file mode 100644 index 000000000..df4a454b6 --- /dev/null +++ b/docs/compatibility.md @@ -0,0 +1,65 @@ +# Compatibility + +This page defines the current documentation and compatibility boundary for +InfiniOps users and contributors. + +## Stable User Entries + +The intended public entries are: + +```cpp +#include +``` + +```python +import infini.ops +``` + +The installed Python wheel is the primary user-facing package surface today. +C++ headers are installed for operator development and integration, but the +consumer-facing C++ package boundary should be kept conservative until covered +by dedicated install-consumer tests. + +## InfiniRT Dependency + +InfiniOps depends on InfiniRT headers and libraries from the configured +`INFINI_RT_ROOT`. Consumers should treat the installed InfiniOps wheel, +InfiniOps headers, and bundled or linked InfiniRT library as a matching set +from the same build. + +## Generated Files + +The build may generate files under `generated/`, including: + +- public operator call instantiations +- generated base classes +- generated PyTorch C++ backend wrappers +- generated Python bindings +- generated metadata for tests + +Generated files are build artifacts. Do not edit them by hand. Change the +source generator, allowlist, or source operator definitions instead. + +## Internal Headers + +Headers under `src/native/**`, `src/torch/**`, and backend-specific runtime +adapters are implementation-facing. They are useful for in-tree development, +tests, and examples, but ordinary users should prefer the public entries above. + +## Backend-Specific Behavior + +Operator availability and supported dtypes, layouts, strides, and implementation +indexes can differ by backend. Tests should document backend-specific skips or +tolerances explicitly instead of hiding them in broad generated results. + +## Source Compatibility + +Follow these rules when changing public or semi-public surfaces: + +- Keep operator signatures stable unless the PR clearly documents the migration + path. +- Preserve generated Python binding names when changing implementation details. +- Update documentation and tests in the same PR when user-visible behavior + changes. +- Treat shared dispatch, wrapper generation, and backend selection changes as + high-risk and validate them across affected platforms. diff --git a/docs/getting-started.md b/docs/getting-started.md new file mode 100644 index 000000000..7e11120d7 --- /dev/null +++ b/docs/getting-started.md @@ -0,0 +1,98 @@ +# Getting Started + +This guide installs InfiniOps and runs a minimal Python operator call. InfiniOps +depends on an installed InfiniRT prefix. + +## Prerequisites + +- C++17 compatible compiler +- CMake 3.18 or newer +- Python 3.10 or newer +- A backend SDK for the target device, such as CUDA Toolkit, DTK, MUSA Toolkit, + Cambricon Neuware, or Ascend CANN +- An installed InfiniRT prefix containing `include/infini/rt.h` and + `lib/libinfinirt.so` + +## Install InfiniRT First + +Build and install InfiniRT into a prefix outside the InfiniOps source tree. The +exact backend options should match the device you want InfiniOps to use. + +```bash +cmake -S /path/to/InfiniRT -B /tmp/infinirt-build \ + -DCMAKE_INSTALL_PREFIX=/path/to/infini-rt-prefix \ + -DWITH_CPU=ON +cmake --build /tmp/infinirt-build -j +cmake --install /tmp/infinirt-build +``` + +The prefix should contain: + +```text +/path/to/infini-rt-prefix/include/infini/rt.h +/path/to/infini-rt-prefix/lib/libinfinirt.so +``` + +## Install InfiniOps + +For a CPU build: + +```bash +python -m pip install . \ + --config-settings=cmake.define.INFINI_RT_ROOT=/path/to/infini-rt-prefix \ + --config-settings=cmake.define.WITH_CPU=ON +``` + +For development, install the optional dependencies: + +```bash +python -m pip install .[dev] \ + --config-settings=cmake.define.INFINI_RT_ROOT=/path/to/infini-rt-prefix \ + --config-settings=cmake.define.WITH_CPU=ON +``` + +InfiniOps can also auto-detect available backends: + +```bash +python -m pip install .[dev] \ + --config-settings=cmake.define.INFINI_RT_ROOT=/path/to/infini-rt-prefix +``` + +Use explicit backend options when auto-detection is not appropriate for the +target machine. + +## Minimal Python Call + +```python +import infini.ops +import torch + +m, n, k = 2, 3, 4 + +x = torch.randn(m, k, device="cpu") +y = torch.randn(k, n, device="cpu") +z = torch.empty(m, n, device="cpu") + +infini.ops.gemm(x, y, z) + +print(z) +``` + +The repository also contains `examples/gemm.py` for a runnable Python example. + +## Minimal C++ Include + +The common public include entry is: + +```cpp +#include +``` + +Most C++ examples in this repository currently exercise in-tree development +headers and backend implementations. A consumer CMake example should be added +as a follow-up once the installed C++ package boundary is documented and tested. + +## Next Steps + +See [Build and Test](build.md) for backend options and smoke test commands, and +[Operators](api/operators.md) for the operator dispatch model.