Storage and observability
Yaostore
Object storage nodes and disk custody
Yaostore manages RustFS nodes, storage pools and member disks. Its controller retains layout and unfinished operations, while node agents perform local configuration, process and disk work. S3 clients reach RustFS directly. Object access is kept separate from initialization, existing-data takeover, pool expansion and retirement.
Add capacity to object storage Register and prepare new member disks, then submit a complete layout that preserves the existing pool order. After affected nodes apply it, continue under the original running or stopped intent. Expansion does not move old objects automatically; read known objects after maintenance and observe any required migration separately.
Give storage maintenance a clear control plane
An operator registers nodes and physical disks, declares pool layout, then applies configuration and authorizes startup as separate actions. The persistent Controller holds registrations, disk custody, desired configuration and unfinished effects. Each local Agent owns its processes, mounts and execution responsibilities.
All nodes share one runtime image and receive distinct assignments at runtime. Adding disk slots or changing pools does not rebuild that image. Clients use the S3 object interface to reach RustFS directly. Its Console manages users, access keys, policies and objects; Yaostore coordinates disk, process and topology maintenance.
Initialize and adopt with different evidence
Initialization targets an explicitly authorized whole disk. The agent fixes the selected device and new identifiers before creating the partition and filesystem, then observes the resulting identities and persistence barriers. An unavailable device, ambiguous identity or unfinished earlier action never becomes evidence of a fresh disk.
Adoption requires independently retained original-member expectations. These cover the physical disk, disk and partition identifiers, filesystem, pool, deployment, drive identity, set and position within that set. Treating a value just read from the disk as its own expectation would also accept the wrong disk. Adoption performs no formatting, and the host owner must still exclude competing writers.
| Task | Required basis | Expected result |
|---|---|---|
| Initialize a new disk | Exact device and explicit destructive authority | New partition, filesystem and custody binding |
| Adopt an existing member | Independent complete original-member expectation | Matched identity with preserved object content |
| Ordinary stop and restart | Existing custody and configuration | Reuse of the same disks without formatting |
From registered disks to the first independently read object
Cluster preparation needs node identities, exact physical disks and a complete pool layout. New disks and existing members require different inputs. A new disk needs explicit destructive initialization authority; an old disk needs independently retained expectations about its original membership. Discovering a device does not authorize either operation.
The first cluster declaration lets every required node apply configuration while stopped. After those actions have settled, authorize startup separately. Then use S3 to write and read an object with independently known content and check the bytes actually returned to the client.
Register the disks and establish their origin
Separate new-disk initialization from old-member adoption, and verify the device identity and authority for the operation.
Apply configuration while stopped
Settle the previous process and related writes, synchronize and apply configuration before allowing a new writer.
Start and read back independently
Use the explicit start operation, then verify known content through the object interface. A management receipt is not a data readback.
Settle old writers before applying a new layout
Initial establishment and recovery of an intact cluster on a new controller first apply a stopped configuration. A separate Start releases running intent. Later changes preserve the existing intent: affected nodes settle old writers and synchronization barriers, all participants apply the required configuration, and only then can running intent resume.
Appending pools preserves the order and width of surviving pools and does not itself move existing data. Removal requires fresh, target-matched native completion observations from every required node. Topology changes allow a maintenance interruption. Pools on one host still share that host’s failure boundary; multiple disks or pools alone do not establish continuous availability.
Separate submission, readiness and completion
Rebalance and decommission commands return a native submission result. Acceptance establishes the native acknowledgement of that request. Progress and completion require a separate status read. Missing progress is distinct from measured zero, and an unavailable result cannot be treated as either rejection or success.
Each action binds the node, boot, agent instance and action identity. Repeated delivery retains the original work and result without renewing its execution budget or formatting again. After a call times out, the actual owner remains responsible until settlement. Controller reopen preserves unfinished responsibility, and loss of contact does not release a disk.
| Result | Established fact | Further observation |
|---|---|---|
| Configuration committed | Desired topology is durable | Actual node configuration and previous writer settlement |
| Node ready | Native probe result for the current binding | Required topology and movement state |
| Native task accepted | Submission acknowledged | Independent progress or terminal-state query |
Check both the outcome and whether work has settled
Investigating unfinished work requires its original node, system boot, agent instance and action ID. Together they identify which execution an observation describes. Re-delivery of the same action reuses that execution. A report from another machine or boot cannot establish completion of the original action.
Outcome and settlement are separate facts. An outcome may be success, failure or unknown. Settlement means the action no longer owns unfinished work. A failure report or an expired wait alone cannot establish that a writer has exited or a mount has been released, so neither authorizes a conflicting task.
Admission condition: the original action has settled
A conflicting resource operation depends on actual settlement and current authority, not on the waiting caller having exited.
Unsupported conclusion: accepted migration means completed movement
A native submission result establishes acceptance only. Removing a pool still requires completion evidence for that exact target.
Check next: the controller reopens or reconnects
Recover durable pending actions and disk custody, then inspect the original executor. Disconnection alone does not release a device.
Explain operation with request and resource signals
Request analysis keeps six populations distinct: S3 operation rates, GET/PUT completion results, successful-request latency, bidirectional HTTP traffic, process CPU and resident memory. HTTP traffic includes administration and probes. CPU is relative to one logical core and can exceed 100%; resident memory is distinct from virtual address space.
The design’s verification concerns include reading back a known object version and its exact bytes after expansion, restart and controller interruption; rejecting an incorrect member expectation; and delaying a previous writer to check that a successor is not released early. Telemetry loss should change observation coverage while preserving admitted storage intent.