How it works

One shard per core

A runtime starts one worker thread per core, and each runs a shard: a run queue, a timer wheel, a buffer pool and a driver, its own handle on the kernel's IO interface. On Linux each worker is pinned to its core.

• A task stays where it was placed. Nothing steals it: no lock or cache line is shared to move it, and its timers and IO stay on one core.
• Placement keeps work near. A task spawned on a worker stays on that worker while it is no busier than the others (a margin you can set); the first spawn from another thread picks the less loaded of two shards.
• A task that blocks holds up only itself. A poll that runs past handoff_after (10 ms by default) hands the shard, with its other tasks, timers and IO, to another thread on the same core (Tasks). The blocked thread still shares that core, so give long work to the blocking pool, or split it with yields.

The driver completes the IO

A readiness-based runtime tells a task when a socket may be ready, and the task then makes the read or write call itself; if another reader got there first, the call would block and the task waits again. Nuclei is a proactor: the task hands its operation to its shard's driver, and the operation completes for it. Only the driver touches a descriptor.

The task's side is the same on all four: it asks, and later takes the result.

Where the bytes go

The kernel may hold an operation's memory after the future that asked is gone, so Nuclei never lends the kernel memory a future borrows:

• Reads land in the shard's pool. The driver reads ahead into a pool buffer; recv or read copies into your slice.
• Writes go to a write-behind. A write is copied into the stream's write-behind buffer and taken at once; the driver writes it out, in order, as the socket takes it.
• The pool has a ceiling. Each worker's buffers hold at most io_buffer_cap bytes. At the cap, IO waits for buffers to come back, so IO memory never grows past it.

So a write that returned means the bytes are Nuclei's to deliver, not that the kernel has them. flush waits until the kernel has every byte and reports a write that failed; shutdown flushes, then ends the write side so the peer sees the end of the stream.

use nuclei::io::{AsyncReadExt, AsyncWriteExt};
use nuclei::net::{TcpListener, TcpStream};

#[nuclei::main]
async fn main() -> std::io::Result<()> {
    let listener = TcpListener::bind("127.0.0.1:0").await?;
    let addr = listener.local_addr()?;
    let reader = nuclei::spawn(async move {
        let (mut stream, _) = listener.accept().await?;
        let mut all = Vec::new();
        stream.read_to_end(&mut all).await?;
        std::io::Result::Ok(all.len())
    });

    let mut stream = TcpStream::connect(addr).await?;
    // Each write is copied into the stream's write-behind and taken at once.
    for _ in 0..1000 {
        stream.write_all(&[b'x'; 1024]).await?;
    }
    // flush waits until the kernel has every byte, and reports a failed write.
    stream.flush().await?;
    // shutdown flushes, then ends the write side: the reader sees the end.
    stream.shutdown().await?;

    assert_eq!(reader.await?, 1000 * 1024);
    Ok(())
}

A write that fails keeps its error: later writes fail with it until a flush reports it. Bytes written behind are delivered even if the handle is dropped first: a task sends them on the handle's own descriptor, for at most write_behind_linger (one minute by default, every setting), and then closes it. Past that the rest is given up, and a TCP stream's connection is reset, so the peer reads ConnectionReset rather than a cut stream that looks whole. abort resets a TCP stream at once (Networking).

A runtime's drop waits for these tasks, running them for at most the same time. One it has not finished by then goes on, on a thread of its own, to its own deadline, as do the ones handed over after the drop.

Lending a buffer

Where the copy matters, lend the driver a buffer of your own. read_owned reads into its spare capacity and gives it back with the count; write_owned writes all of it and gives it back once the kernel took the last byte. Each takes any buffer type that implements IoBuf (writes) or IoBufMut (reads), a Vec<u8> among them, and gives the same type back. TCP and Unix streams, pipes and files have both, and read_vectored_owned and write_vectored_owned lend several buffers in one call (IO utilities).

use nuclei::net::{TcpListener, TcpStream};

#[nuclei::main]
async fn main() -> std::io::Result<()> {
    let listener = TcpListener::bind("127.0.0.1:0").await?;
    let addr = listener.local_addr()?;
    let server = nuclei::spawn(async move {
        let (stream, _) = listener.accept().await?;
        // Lend the driver this vector: the kernel reads into its spare capacity.
        let (n, buf) = stream.read_owned(Vec::with_capacity(64)).await;
        n?;
        // Lend it again for the reply; it comes back once the kernel took it all.
        let (sent, _buf) = stream.write_owned(buf).await;
        sent
    });

    let client = TcpStream::connect(addr).await?;
    let (sent, _) = client.write_owned(b"lent, not copied".to_vec()).await;
    sent?;
    let (n, reply) = client.read_owned(Vec::with_capacity(64)).await;
    assert_eq!(&reply[..n?], b"lent, not copied");
    server.await?;
    Ok(())
}

Dropping a future halfway

Async code drops futures all the time: a timeout fires, a token is cancelled, a select! takes another branch. No byte is lost when it does:

• A read dropped halfway leaves what the driver already read for the next read.
• A lent read (read_owned) dropped halfway hands the bytes read into its buffer to the next read, ahead of anything after them.
• A lent write (write_owned) dropped halfway reaches the stream whole or not at all: withdrawn if none of it went out, finished by the driver if some did. On Windows a TCP write is always finished.

No allocation in steady state

Once warm, these operations make no call into the allocator. The crate's tests measure it with a counting allocator (tests/zero_alloc.rs), on a runtime that runs everything on one thread:

• a TCP round trip; UDP datagrams, to an address and on a connected socket; file reads and writes;
• a local spawn; a spawn onto a worker, on the spawning thread; a job on the blocking pool;
• a JoinSet spawn and join; a oneshot channel.

A timer takes a slot of its shard's wheel, and arming and cancelling reuse the slots once the wheel has grown to the most timers alive at once.

Checked, not assumed

The concurrent protocols are model-checked with TLA+: the task's states, the park and wake handshake, an IO source's sides, the ring's operations and the sync primitives' queues. TLC tries every order of every thread's steps for small sizes, and each model has mutations that must break it. Lean proves the arithmetic under them (counts, state words, generations) for every size. Loom runs the real code under every interleaving it reaches, and CI runs the suite under AddressSanitizer on io_uring and on epoll. proofs/README.md says what each check covers and what it leaves out.