Native Modules and .xulib Specification
In systems programming, integrating with mature C/C++ native codebases is a core requirement. Xu delivers zero-abstraction-overhead, type-safe native interop. Developers often ask: "Why does Xu introduce .xulib alongside standard universal .a (static archive) files?"
This technical guide details Xu's native interop architecture, the distinct roles of universal static archives versus .xulib containers, the complete multi-type ABI mapping table, and the end-to-end 7 development actions.
1. Core Architecture: Universal .a vs .xulib Containers
- Standard universal
.a(Unix static archives) are first-class peers: Usexu native compileorxu native archiveto include the Xu interface metadata alongside native objects. Onmacos-arm64andmacos-x64, link viaxu build main.xu module.aor declare it undernative:inxu.xun. It is statically linked by the system linker. .xulibis a multi-target slice distribution container: It is not a proprietary machine-code format. Internally, it bundles standard platform.astatic slices, augmented by a strongly typed contract (JSON Manifest) and SHA-256 integrity digests.
One-step native linking currently supports only macos-arm64 and macos-x64 and requires the macOS C toolchain. Other targets require --emit-obj, conversion of XUOBJ to a platform object, and external system linking. A multi-target container does not extend this linking support. The seven-step example below uses macOS slices.
| Dimension | Universal .a Static Archive | Portable .xulib Container |
|---|---|---|
| Primary Scope | Single-target development, local debugging, private C integrations | Cross-platform ecosystem packages (e.g. packages/sqlite) |
| Target Architecture | Single OS/Arch slice (e.g. only linux-x64 or macos-arm64) | Multi-target, multi-architecture bundle in a single file |
| Cross-Platform Linking | Requires custom build scripts to branch per OS/Arch | Compiler automatically selects and extracts matching slice based on --target |
| Type Safety | Archives built by xu native compile/archive include XUOBJ export signatures | Embedded JSON Manifest & XUOBJ metadata for compile-time checking |
| Tamper Proofing | No built-in cryptographic validation | Every target slice records an embedded SHA-256 digest |
2. Multi-Type ABI Mapping Matrix
Under the abi: "xu-c-1" protocol, Xu passes scalar values directly through machine general-purpose registers:
| Xu Type | C Mapping | Semantics & Passing Conventions |
|---|---|---|
int | xu_int (int64_t) | 64-bit signed integer for arithmetic, indices, and offsets |
f64 | 64-bit float (IEEE-754) | Double-precision float passed via scalar register bit pattern. Converted via inline helpers |
bool | uint8_t / bool | Boolean flag (1 represents true, 0 represents false) |
i32 | int32_t | 32-bit signed integer for OS status codes and clamped ranges |
u32 | uint32_t | 32-bit unsigned integer for bitwise shifts, masks, and hashes |
u8 | uint8_t | 8-bit unsigned byte for protocol parsing, bytecodes, and pixel channels |
void | void | Procedures without returns, for state initialization and side-effects |
2.1 Synchronous borrowed i64 spans: xu-c-2
Xu 0.11.3 adds the opt-in abi: "xu-c-2" contract. A manifest parameter i64_span maps to Xu &[]int and a pointer to xu_i64_span; i64_span_mut maps to $&[]int and a pointer to xu_i64_span_mut. The matching include/xu_native.h defines the C layouts: an eight-byte element count followed by a data pointer.
Each synchronous call receives a separate temporary descriptor. Native code must not retain the descriptor or data pointer, change descriptor fields, resize the array or take ownership. A writable span permits element writes only. Empty spans may have a null data pointer. Capacity and Xu's private descriptor remain inside the runtime.
C2 currently supports hosted 64-bit targets, 64-bit int scalar parameters, and int/void results. It does not define floating signatures, general callbacks or borrowed results. Platform argument registers, stack arguments and Win64 shadow space are handled by the adapter; this does not extend the one-step linker support described above. Use a Xu wrapper when a native function must be passed as a function value.
C2 selects XUOBJ version 2 and xulib-2, with container version 2 at byte offset 7. Older products explicitly reject the new format; C1 libraries remain supported. The following binary diagram describes the C1 container layout. Container selection and slice extraction happen during the build; C2 still performs its call-scoped descriptor and argument staging at runtime.
3. .xulib Binary Layout
.xulib is a self-describing multi-architecture static library container (specification xulib-1). It follows a compact linear segment layout consisting of a magic signature, fixed header, manifest contract, slice index table, and multi-architecture payload slices:
module identifier, abi: "xu-c-1", format: "xulib-1", and slices target SHA-256 digests; export signatures reside in each slice’s XUOBJ metadatatarget_name, kind (archive/object), file_offset & file_size, and sha256 integrity digestbridge.o + xu.o __DATA,__xumeta).o + xu.o .xumeta).obj + xu.o .xumeta)Complete .xulib binary container specification table:
| Offset Range | Field Name | Type & Size | Specification & Semantic Description |
|---|---|---|---|
| Segment 1: Fixed Container Header (15 Bytes) | |||
0x00..0x06 (7 Bytes) |
Magic Header | char[7] |
Fixed "XULIB\x01\x00" identifying a valid container and verifying major format version. |
0x07..0x08 (2 Bytes) |
Version | u16 (little-endian) |
Container format minor version; currently fixed to 1. |
0x09..0x0A (2 Bytes) |
Flags | u16 (bitmask) |
Reserved for future container features (e.g., compression, digital signatures); default 0. |
0x0B..0x0E (4 Bytes) |
Manifest Length | u32 (little-endian) |
Byte length $N$ of trailing JSON contract payload. |
| Segment 2: JSON Contract Manifest (N Bytes) | |||
0x0F..0x0F+N ($N$ Bytes) |
JSON Manifest | UTF-8 Compact Text |
Top-level container metadata written by xu native pack:• module: globally unique module namespace identifier (e.g. "native_add")• abi: native interface protocol specification (e.g. "xu-c-1")• format: container format identifier ("xulib-1")• slices: target platform slice names and corresponding SHA-256 digests
|
| Segment 3: Slice Index Table | |||
Trailing 0x0F+N (4 Bytes) |
Slice Count | u32 (little-endian) |
Count of contiguous target architecture slice entries in the index table ($M$). |
| Contiguous Entries | Slice Index Table Entries | Array of Variable Structs |
Contiguous slice entries, each containing: • target_len (u16) + target_name (e.g. "macos-arm64", "linux-x64")• kind_len (u8) + kind ("archive" or "object")• file_offset (u64) + file_size (u64) absolute offset & length• sha256 (32 Bytes) binary integrity digest
|
| Segment 4: Multi-Architecture Binary Payloads | |||
| Offset from Index | macOS Slice (Mach-O) | Standard Unix .a |
Mach-O native object files (e.g. bridge.o) + xu.o (with __DATA,__xumeta section). |
| Offset from Index | Linux Slice (ELF) | Standard Unix .a |
Linux ELF native object files + xu.o (with .xumeta section). |
| Offset from Index | Windows Slice (COFF) | Standard .a / .lib |
Windows COFF native object files + xu.o (with .xumeta section). |
Clever Dual Compatibility Design: Each platform slice is fundamentally a standard .a static archive. It bundles native business logic objects (e.g. sqlite3.o, bridge.o) compiled by external toolchains (such as C, C++, Rust, or assembly) together with a fully compliant native metadata object member (xu.o). The host platform linker directly links the native machine code symbols, while the Xu compiler inspects the XUOBJ type contract embedded in the dedicated .xumeta section (in Mach-O: __DATA,__xumeta) for compile-time type verification, achieving seamless zero-conflict coexistence.
The input native.json defines exports; their typed interface is embedded in each archive’s xu.o XUOBJ metadata. The product xu native pack does not copy exports into the top-level manifest. The source-tree Python helper scripts/xunative.py copies the input manifest, so its output also retains exports. Consumers must read slice metadata rather than require top-level exports.
4. The Complete 7 Development Actions
Action 1: Write C Bridge Source (bridge.c)
Include xu_native.h and wrap export declarations with XU_NATIVE_SYMBOL(...) to reconcile Mach-O leading underscore conventions with ELF/COFF:
#include "xu_native.h"
#include <stdint.h>
#include <stdbool.h>
#include <string.h>
// 1. int (64-bit integer)
xu_int XU_NATIVE_SYMBOL(xu_native_add_add)(xu_int a, xu_int b) {
return a + b;
}
// 2. f64 (64-bit float bitwise mapping)
static inline double xu_to_double(xu_int bits) {
double v; memcpy(&v;, &bits;, sizeof(double)); return v;
}
static inline xu_int double_to_xu(double v) {
xu_int b; memcpy(&b;, &v;, sizeof(xu_int)); return b;
}
xu_int XU_NATIVE_SYMBOL(xu_native_add_scale_f64)(xu_int val_bits, xu_int factor_bits) {
return double_to_xu(xu_to_double(val_bits) * xu_to_double(factor_bits));
}
// 3. bool (boolean)
uint8_t XU_NATIVE_SYMBOL(xu_native_add_is_even)(xu_int n) {
return (n % 2 == 0) ? 1 : 0;
}
// 4. i32 (32-bit signed int clamp)
int32_t XU_NATIVE_SYMBOL(xu_native_add_clamp_i32)(int32_t val, int32_t min_v, int32_t max_v) {
if (val < min_v) return min_v;
if (val > max_v) return max_v;
return val;
}
// 5. u32 (32-bit unsigned int bitwise rotate)
uint32_t XU_NATIVE_SYMBOL(xu_native_add_rotate_left_u32)(uint32_t val, uint32_t shift) {
shift &= 31;
if (shift == 0) return val;
return (val << shift) | (val >> (32 - shift));
}
// 6. u8 (8-bit unsigned byte mask)
uint8_t XU_NATIVE_SYMBOL(xu_native_add_mask_u8)(uint8_t b, uint8_t mask) {
return (uint8_t)(b & mask);
}
// 7. void (stateful procedure with side-effects)
static xu_int g_counter = 0;
void XU_NATIVE_SYMBOL(xu_native_add_reset_counter)(void) { g_counter = 0; }
void XU_NATIVE_SYMBOL(xu_native_add_add_to_counter)(xu_int delta) { g_counter += delta; }
xu_int XU_NATIVE_SYMBOL(xu_native_add_get_counter)(void) { return g_counter; }
Action 2: Declare Type Contract Manifest (native.json)
{
"module": "native_add",
"abi": "xu-c-1",
"exports": [
{"name": "add", "params": ["int", "int"], "ret": "int"},
{"name": "scale_f64", "params": ["f64", "f64"], "ret": "f64"},
{"name": "is_even", "params": ["int"], "ret": "bool"},
{"name": "clamp_i32", "params": ["i32", "i32", "i32"], "ret": "i32"},
{"name": "rotate_left_u32", "params": ["u32", "u32"], "ret": "u32"},
{"name": "mask_u8", "params": ["u8", "u8"], "ret": "u8"},
{"name": "reset_counter", "params": [], "ret": "void"},
{"name": "add_to_counter", "params": ["int"], "ret": "void"},
{"name": "get_counter", "params": [], "ret": "int"}
]
}
Action 3: Compile Target Slices (xu native compile)
mkdir -p native
# macOS Apple Silicon (ARM64)
xu native compile \
--manifest native.json --target macos-arm64 bridge.c \
-o native/native_add-arm64.a
# macOS Intel (x86_64)
xu native compile \
--manifest native.json --target macos-x64 bridge.c \
-o native/native_add-x64.a
Action 4: Pack Multi-Target Archive (xu native pack)
xu native pack \
--manifest native.json \
--slice macos-arm64=native/native_add-arm64.a \
--slice macos-x64=native/native_add-x64.a \
-o native/native_add.xulib
You can also run xu native inspect native/native_add.xulib to verify container slices and SHA-256 digests, or extract slices with xu native extract.
Action 5: Declare Native Dependency (xu.xun)
module: my_project
version: !ver 0.1.0
xu: !ver 0.13.1
native:
native_add: file:./native/native_add.xulib
Action 6: Import & Call from Xu Code (main.xu)
@main
#native_add
fn main() int {
println("=== Xu <-> Native C Multi-Type Interop Demo ===")
// 1. int
sum := native_add.add(20, 22)
println("1. int add(20, 22) = " + str(sum))
// 2. f64
scaled := native_add.scale_f64(1.5, 3.0)
println("2. f64 scale_f64(1.5, 3.0) * 10 = " + str(int(scaled * 10.0)))
// 3. bool
println("3. bool is_even(42) = " + str(native_add.is_even(42)))
// 4. i32
clamped := native_add.clamp_i32(i32(150), i32(0), i32(100))
println("4. i32 clamp_i32(150, 0, 100) = " + str(int(clamped)))
// 5. u32
rotated := native_add.rotate_left_u32(u32(1), u32(4))
println("5. u32 rotate_left_u32(1, 4) = " + str(int(rotated)))
// 6. u8
masked := native_add.mask_u8(u8(255), u8(15))
println("6. u8 mask_u8(255, 15) = " + str(int(masked)))
// 7. void
native_add.reset_counter()
native_add.add_to_counter(100)
c := native_add.get_counter()
println("7. void state counter = " + str(c))
println("=== All Native Type Assertions Passed ===")
<< 0
}
Action 7: Compile & Verify Execution (macOS)
xu run .
The console prints the verified results:
=== Xu <-> Native C Multi-Type Interop Demo ===
1. int add(20, 22) = 42
2. f64 scale_f64(1.5, 3.0) * 10 = 45
3. bool is_even(42) = true
4. i32 clamp_i32(150, 0, 100) = 100
5. u32 rotate_left_u32(1, 4) = 16
6. u8 mask_u8(255, 15) = 15
7. void state counter = 100
=== All Native Type Assertions Passed ===
External linking on Linux x64
With Clang and an archive tool installed, prepare a linux-x64 archive using xu native compile --manifest native.json --target linux-x64 bridge.c -o native_add-linux.a. Using the main.xu above and that archive, emit XUOBJ, convert it with the matching SDK’s internal _elf_o helper, then link externally:
xu build main.xu native_add-linux.a --target linux-x64 --emit-obj -o main.xuobj
xu _elf_o main.xuobj main.o
cc -nostartfiles -no-pie main.o native_add-linux.a -o app
./app
Xu supplies its own entry point, so this Linux link omits the C startup files. Windows and other targets need their own object conversion and linker entry-point/runtime options; the Linux command is specific to Linux x64.
5. Frequently Asked Questions (FAQ)
Q: Can I use single-platform .a directly without creating a .xulib?
A: Yes, on the supported macOS targets. You can point directly to native_add: file:./native/native_add-arm64.a in xu.xun or pass it on the command line: xu build main.xu ./native/native_add-arm64.a. Ensure the archive contains the xu.o metadata member produced by xu native compile or xu native archive.
Q: Does .xulib introduce any runtime overhead?
A: The container is processed at build time and needs no runtime unpacker. The selected native slice is linked into the application. C2 calls also prepare borrowed descriptors and platform arguments; container extraction does not remove that ABI adapter work.
Modules and dependencies · External packages · CLI reference