XU XUv0.13.1
中文

Docs

Native Modules and .xulib Specification

In systems programming, integrating with mature C/C++ native codebases is a core requirement. Xu delivers zero-abstraction-overhead, type-safe native interop. Developers often ask: "Why does Xu introduce .xulib alongside standard universal .a (static archive) files?"

This technical guide details Xu's native interop architecture, the distinct roles of universal static archives versus .xulib containers, the complete multi-type ABI mapping table, and the end-to-end 7 development actions.

1. Core Architecture: Universal .a vs .xulib Containers

One-step native linking currently supports only macos-arm64 and macos-x64 and requires the macOS C toolchain. Other targets require --emit-obj, conversion of XUOBJ to a platform object, and external system linking. A multi-target container does not extend this linking support. The seven-step example below uses macOS slices.

DimensionUniversal .a Static ArchivePortable .xulib Container
Primary ScopeSingle-target development, local debugging, private C integrationsCross-platform ecosystem packages (e.g. packages/sqlite)
Target ArchitectureSingle OS/Arch slice (e.g. only linux-x64 or macos-arm64)Multi-target, multi-architecture bundle in a single file
Cross-Platform LinkingRequires custom build scripts to branch per OS/ArchCompiler automatically selects and extracts matching slice based on --target
Type SafetyArchives built by xu native compile/archive include XUOBJ export signaturesEmbedded JSON Manifest & XUOBJ metadata for compile-time checking
Tamper ProofingNo built-in cryptographic validationEvery target slice records an embedded SHA-256 digest

2. Multi-Type ABI Mapping Matrix

Under the abi: "xu-c-1" protocol, Xu passes scalar values directly through machine general-purpose registers:

Xu TypeC MappingSemantics & Passing Conventions
intxu_int (int64_t)64-bit signed integer for arithmetic, indices, and offsets
f6464-bit float (IEEE-754)Double-precision float passed via scalar register bit pattern. Converted via inline helpers
booluint8_t / boolBoolean flag (1 represents true, 0 represents false)
i32int32_t32-bit signed integer for OS status codes and clamped ranges
u32uint32_t32-bit unsigned integer for bitwise shifts, masks, and hashes
u8uint8_t8-bit unsigned byte for protocol parsing, bytecodes, and pixel channels
voidvoidProcedures without returns, for state initialization and side-effects

2.1 Synchronous borrowed i64 spans: xu-c-2

Xu 0.11.3 adds the opt-in abi: "xu-c-2" contract. A manifest parameter i64_span maps to Xu &[]int and a pointer to xu_i64_span; i64_span_mut maps to $&[]int and a pointer to xu_i64_span_mut. The matching include/xu_native.h defines the C layouts: an eight-byte element count followed by a data pointer.

Each synchronous call receives a separate temporary descriptor. Native code must not retain the descriptor or data pointer, change descriptor fields, resize the array or take ownership. A writable span permits element writes only. Empty spans may have a null data pointer. Capacity and Xu's private descriptor remain inside the runtime.

C2 currently supports hosted 64-bit targets, 64-bit int scalar parameters, and int/void results. It does not define floating signatures, general callbacks or borrowed results. Platform argument registers, stack arguments and Win64 shadow space are handled by the adapter; this does not extend the one-step linker support described above. Use a Xu wrapper when a native function must be passed as a function value.

C2 selects XUOBJ version 2 and xulib-2, with container version 2 at byte offset 7. Older products explicitly reject the new format; C1 libraries remain supported. The following binary diagram describes the C1 container layout. Container selection and slice extraction happen during the build; C2 still performs its call-scoped descriptor and argument staging at runtime.

3. .xulib Binary Layout

.xulib is a self-describing multi-architecture static library container (specification xulib-1). It follows a compact linear segment layout consisting of a magic signature, fixed header, manifest contract, slice index table, and multi-architecture payload slices:

.xulib Physical Memory & Disk Binary Layout
SPEC xulib-1
0x00..0x06 (7B)
Magic: "XULIB\x01\x00"
Magic signature and major version check
0x07..0x08 (2B)
Version = 1
u16 minor version
0x09..0x0A (2B)
Flags = 0
u16 bitmask flags
0x0B..0x0E (4B)
Manifest Len = N
u32 little-endian manifest length
0x0F..0x0F+N (N Bytes UTF-8)
JSON Manifest (Interface Contract & Hash Manifest)
Contains module identifier, abi: "xu-c-1", format: "xulib-1", and slices target SHA-256 digests; export signatures reside in each slice’s XUOBJ metadata
0x0F+N (4B)
Slice Count = M
u32 little-endian slice count
Variable Length Entries
Slice Index Table (Target Architectures Index Array)
Each entry records: target_name, kind (archive/object), file_offset & file_size, and sha256 integrity digest
macOS Slice Offset
macOS Slice (Mach-O)
Standard Unix .a (bridge.o + xu.o __DATA,__xumeta)
Linux Slice Offset
Linux Slice (ELF)
Standard Unix .a (native .o + xu.o .xumeta)
Windows Slice Offset
Windows Slice (COFF)
Standard .a / .lib (native .obj + xu.o .xumeta)

Complete .xulib binary container specification table:

Offset Range Field Name Type & Size Specification & Semantic Description
Segment 1: Fixed Container Header (15 Bytes)
0x00..0x06 (7 Bytes) Magic Header char[7] Fixed "XULIB\x01\x00" identifying a valid container and verifying major format version.
0x07..0x08 (2 Bytes) Version u16 (little-endian) Container format minor version; currently fixed to 1.
0x09..0x0A (2 Bytes) Flags u16 (bitmask) Reserved for future container features (e.g., compression, digital signatures); default 0.
0x0B..0x0E (4 Bytes) Manifest Length u32 (little-endian) Byte length $N$ of trailing JSON contract payload.
Segment 2: JSON Contract Manifest (N Bytes)
0x0F..0x0F+N ($N$ Bytes) JSON Manifest UTF-8 Compact Text Top-level container metadata written by xu native pack:
• module: globally unique module namespace identifier (e.g. "native_add")
• abi: native interface protocol specification (e.g. "xu-c-1")
• format: container format identifier ("xulib-1")
• slices: target platform slice names and corresponding SHA-256 digests
Segment 3: Slice Index Table
Trailing 0x0F+N (4 Bytes) Slice Count u32 (little-endian) Count of contiguous target architecture slice entries in the index table ($M$).
Contiguous Entries Slice Index Table Entries Array of Variable Structs Contiguous slice entries, each containing:
• target_len (u16) + target_name (e.g. "macos-arm64", "linux-x64")
• kind_len (u8) + kind ("archive" or "object")
• file_offset (u64) + file_size (u64) absolute offset & length
• sha256 (32 Bytes) binary integrity digest
Segment 4: Multi-Architecture Binary Payloads
Offset from Index macOS Slice (Mach-O) Standard Unix .a Mach-O native object files (e.g. bridge.o) + xu.o (with __DATA,__xumeta section).
Offset from Index Linux Slice (ELF) Standard Unix .a Linux ELF native object files + xu.o (with .xumeta section).
Offset from Index Windows Slice (COFF) Standard .a / .lib Windows COFF native object files + xu.o (with .xumeta section).

Clever Dual Compatibility Design: Each platform slice is fundamentally a standard .a static archive. It bundles native business logic objects (e.g. sqlite3.o, bridge.o) compiled by external toolchains (such as C, C++, Rust, or assembly) together with a fully compliant native metadata object member (xu.o). The host platform linker directly links the native machine code symbols, while the Xu compiler inspects the XUOBJ type contract embedded in the dedicated .xumeta section (in Mach-O: __DATA,__xumeta) for compile-time type verification, achieving seamless zero-conflict coexistence.

The input native.json defines exports; their typed interface is embedded in each archive’s xu.o XUOBJ metadata. The product xu native pack does not copy exports into the top-level manifest. The source-tree Python helper scripts/xunative.py copies the input manifest, so its output also retains exports. Consumers must read slice metadata rather than require top-level exports.

4. The Complete 7 Development Actions

Action 1: Write C Bridge Source (bridge.c)

Include xu_native.h and wrap export declarations with XU_NATIVE_SYMBOL(...) to reconcile Mach-O leading underscore conventions with ELF/COFF:

#include "xu_native.h"
#include <stdint.h>
#include <stdbool.h>
#include <string.h>

// 1. int (64-bit integer)
xu_int XU_NATIVE_SYMBOL(xu_native_add_add)(xu_int a, xu_int b) {
    return a + b;
}

// 2. f64 (64-bit float bitwise mapping)
static inline double xu_to_double(xu_int bits) {
    double v; memcpy(&v;, &bits;, sizeof(double)); return v;
}
static inline xu_int double_to_xu(double v) {
    xu_int b; memcpy(&b;, &v;, sizeof(xu_int)); return b;
}
xu_int XU_NATIVE_SYMBOL(xu_native_add_scale_f64)(xu_int val_bits, xu_int factor_bits) {
    return double_to_xu(xu_to_double(val_bits) * xu_to_double(factor_bits));
}

// 3. bool (boolean)
uint8_t XU_NATIVE_SYMBOL(xu_native_add_is_even)(xu_int n) {
    return (n % 2 == 0) ? 1 : 0;
}

// 4. i32 (32-bit signed int clamp)
int32_t XU_NATIVE_SYMBOL(xu_native_add_clamp_i32)(int32_t val, int32_t min_v, int32_t max_v) {
    if (val < min_v) return min_v;
    if (val > max_v) return max_v;
    return val;
}

// 5. u32 (32-bit unsigned int bitwise rotate)
uint32_t XU_NATIVE_SYMBOL(xu_native_add_rotate_left_u32)(uint32_t val, uint32_t shift) {
    shift &= 31;
    if (shift == 0) return val;
    return (val << shift) | (val >> (32 - shift));
}

// 6. u8 (8-bit unsigned byte mask)
uint8_t XU_NATIVE_SYMBOL(xu_native_add_mask_u8)(uint8_t b, uint8_t mask) {
    return (uint8_t)(b & mask);
}

// 7. void (stateful procedure with side-effects)
static xu_int g_counter = 0;
void XU_NATIVE_SYMBOL(xu_native_add_reset_counter)(void) { g_counter = 0; }
void XU_NATIVE_SYMBOL(xu_native_add_add_to_counter)(xu_int delta) { g_counter += delta; }
xu_int XU_NATIVE_SYMBOL(xu_native_add_get_counter)(void) { return g_counter; }

Action 2: Declare Type Contract Manifest (native.json)

{
  "module": "native_add",
  "abi": "xu-c-1",
  "exports": [
    {"name": "add", "params": ["int", "int"], "ret": "int"},
    {"name": "scale_f64", "params": ["f64", "f64"], "ret": "f64"},
    {"name": "is_even", "params": ["int"], "ret": "bool"},
    {"name": "clamp_i32", "params": ["i32", "i32", "i32"], "ret": "i32"},
    {"name": "rotate_left_u32", "params": ["u32", "u32"], "ret": "u32"},
    {"name": "mask_u8", "params": ["u8", "u8"], "ret": "u8"},
    {"name": "reset_counter", "params": [], "ret": "void"},
    {"name": "add_to_counter", "params": ["int"], "ret": "void"},
    {"name": "get_counter", "params": [], "ret": "int"}
  ]
}

Action 3: Compile Target Slices (xu native compile)

mkdir -p native

# macOS Apple Silicon (ARM64)
xu native compile \
  --manifest native.json --target macos-arm64 bridge.c \
  -o native/native_add-arm64.a

# macOS Intel (x86_64)
xu native compile \
  --manifest native.json --target macos-x64 bridge.c \
  -o native/native_add-x64.a

Action 4: Pack Multi-Target Archive (xu native pack)

xu native pack \
  --manifest native.json \
  --slice macos-arm64=native/native_add-arm64.a \
  --slice macos-x64=native/native_add-x64.a \
  -o native/native_add.xulib

You can also run xu native inspect native/native_add.xulib to verify container slices and SHA-256 digests, or extract slices with xu native extract.

Action 5: Declare Native Dependency (xu.xun)

module: my_project
version: !ver 0.1.0
xu: !ver 0.13.1

native:
  native_add: file:./native/native_add.xulib

Action 6: Import & Call from Xu Code (main.xu)

@main

#native_add

fn main() int {
    println("=== Xu <-> Native C Multi-Type Interop Demo ===")

    // 1. int
    sum := native_add.add(20, 22)
    println("1. int add(20, 22) = " + str(sum))

    // 2. f64
    scaled := native_add.scale_f64(1.5, 3.0)
    println("2. f64 scale_f64(1.5, 3.0) * 10 = " + str(int(scaled * 10.0)))

    // 3. bool
    println("3. bool is_even(42) = " + str(native_add.is_even(42)))

    // 4. i32
    clamped := native_add.clamp_i32(i32(150), i32(0), i32(100))
    println("4. i32 clamp_i32(150, 0, 100) = " + str(int(clamped)))

    // 5. u32
    rotated := native_add.rotate_left_u32(u32(1), u32(4))
    println("5. u32 rotate_left_u32(1, 4) = " + str(int(rotated)))

    // 6. u8
    masked := native_add.mask_u8(u8(255), u8(15))
    println("6. u8 mask_u8(255, 15) = " + str(int(masked)))

    // 7. void
    native_add.reset_counter()
    native_add.add_to_counter(100)
    c := native_add.get_counter()
    println("7. void state counter = " + str(c))

    println("=== All Native Type Assertions Passed ===")
    << 0
}

Action 7: Compile & Verify Execution (macOS)

xu run .

The console prints the verified results:

=== Xu <-> Native C Multi-Type Interop Demo ===
1. int add(20, 22) = 42
2. f64 scale_f64(1.5, 3.0) * 10 = 45
3. bool is_even(42) = true
4. i32 clamp_i32(150, 0, 100) = 100
5. u32 rotate_left_u32(1, 4) = 16
6. u8 mask_u8(255, 15) = 15
7. void state counter = 100
=== All Native Type Assertions Passed ===

External linking on Linux x64

With Clang and an archive tool installed, prepare a linux-x64 archive using xu native compile --manifest native.json --target linux-x64 bridge.c -o native_add-linux.a. Using the main.xu above and that archive, emit XUOBJ, convert it with the matching SDK’s internal _elf_o helper, then link externally:

xu build main.xu native_add-linux.a --target linux-x64 --emit-obj -o main.xuobj
xu _elf_o main.xuobj main.o
cc -nostartfiles -no-pie main.o native_add-linux.a -o app
./app

Xu supplies its own entry point, so this Linux link omits the C startup files. Windows and other targets need their own object conversion and linker entry-point/runtime options; the Linux command is specific to Linux x64.

5. Frequently Asked Questions (FAQ)

Q: Can I use single-platform .a directly without creating a .xulib?
A: Yes, on the supported macOS targets. You can point directly to native_add: file:./native/native_add-arm64.a in xu.xun or pass it on the command line: xu build main.xu ./native/native_add-arm64.a. Ensure the archive contains the xu.o metadata member produced by xu native compile or xu native archive.

Q: Does .xulib introduce any runtime overhead?
A: The container is processed at build time and needs no runtime unpacker. The selected native slice is linked into the application. C2 calls also prepare borrowed descriptors and platform arguments; container extraction does not remove that ABI adapter work.

Modules and dependencies · External packages · CLI reference