---
title: "Storage"
description: "OpenPit policies often need internal state - accumulated P&L, rate-limit timestamps, reserved margin, position counters. The Storage abstraction keeps."
---
# Storage
OpenPit policies often need internal state - accumulated P&L, rate-limit
timestamps, reserved margin, position counters. The `Storage` abstraction
keeps that state correct in any synchronization environment the SDK
supports, with zero overhead when none is needed and full thread-safety
when the embedding requires it.
## Why a dedicated abstraction
A custom policy that stores state in a plain map handles only the
single-thread case. The moment calls cross threads - even sequentially -
the policy needs a mechanism that establishes happens-before ordering
between writes on one thread and reads on another. `Storage` provides
that ordering without making the policy author design cross-thread
coordination.
Cooperative scheduling adds the same problem in a less obvious form: a
scheduled task may resume on a different OS thread than the one it suspended
on. Calls may still be sequential at the host layer, but the happens-before
guarantee is still required. The engine handle carries the matching
capability through the selected sync policy (Full / Account / No sync).
Across the language bindings, every storage created by a policy is
**account-keyed**: the top-level key carries the `AccountId`. The per-account
value can be any structure the policy needs (counter, sliding-window log,
nested map). This sharding is part of the public storage contract and is
independent from the chosen sync mode.
`Storage` solves the policy-state side of these cases:
- the synchronization policy is selected once when the storage is created;
- every read or write goes through a scoped access handle that owns the
appropriate rights for the duration of access;
- user code only asks the storage to read or write a key;
- under the no-sync policy, access compiles down to direct map operations
with no synchronization overhead.
## Synchronization policies
The storage picks one of three built-in policies at construction time.
The policy is part of the storage's type, so runtime overhead matches
the chosen guarantees and nothing more.
### No-sync policy
The host keeps every call on one thread. The storage performs no
cross-thread coordination, and sharing it across threads is rejected at
the language level. Lowest-overhead choice: operations compile down to
direct lookups.
### Full-sync policy
Any thread may enter for any key in any order. The storage serializes reads
and writes so the state stays consistent under concurrent load. Safe
default when the host does not constrain the call pattern.
### Account sync
The caller commits to processing each account through a single chain: one
queue or one worker at a time. The same account must never be processed by
two threads in parallel; different accounts may run concurrently with no
coordination.
The storage coordinates the boundary between adding/removing accounts and
accessing per-account state, but does not serialize per-account access -
the single-chain-per-account discipline does. Throughput scales with the
number of independent accounts.
### Custom
The Rust API exposes the policy abstraction so that custom
synchronization strategies can be plugged in (for example, a sharded
coordinator or a lock-free data structure).
### Cost model
- **Direct Rust** - No-sync: zero synchronization overhead. Account sync:
one lock acquire on the index per access. Full sync: two lock acquires
(index and values) per access. The engine handle adds an atomic reference
only for modes that produce a `Send` handle (Full and Account); No-sync
uses a non-atomic reference.
- **Language bindings** - every mode goes through a thread-portable engine
handle and a runtime sync-mode dispatch. The fixed binding overhead is
small (one atomic handle reference plus a runtime dispatch) compared to
the per-mode storage cost above.
In all modes, business-logic costs (policy evaluation, map operations)
dominate per-call wall time. Locking overhead matters only at very high
throughput.
## Engine integration
Each engine builder owns a storage builder whose policy type matches the
engine's synchronization policy. Pass a reference to that storage builder
to a custom policy's constructor; the policy creates one storage per
internal table and keeps it for the engine's lifetime.
Application code cannot create a `StorageBuilder` directly. The engine is
the only owner of the builder, and policies that need storage must receive
`builder.storage_builder()` during initialization. The sync policy is fixed
once, when the engine is configured - every storage built from that builder
shares the same locking guarantees.
## Example: Custom Policy with Storage
Rust
```rust
use openpit::param::{AccountId, Asset, Pnl};
use openpit::storage::{LockingPolicyFactory, Storage, StorageBuilder};
pub struct MyPolicy
where
StorageLockingPolicyFactory: LockingPolicyFactory,
{
realized: Storage<
(AccountId, Asset),
Pnl,
StorageLockingPolicyFactory::Policy,
>,
}
impl MyPolicy
where
StorageLockingPolicyFactory: LockingPolicyFactory
+ openpit::storage::CreateStorageFor<(AccountId, Asset)>,
{
pub fn new(
storage_builder: &StorageBuilder,
) -> Self {
Self {
realized: storage_builder.create_for_bound_key(),
}
}
pub fn record_pnl(
&self,
account: AccountId,
settlement: Asset,
delta: Pnl,
) {
self.realized.with_mut(
(account, settlement),
|| Pnl::ZERO,
|entry, _is_new| {
if let Ok(updated) = entry.checked_add(delta) {
*entry = updated;
}
},
);
}
pub fn current_pnl(
&self,
account: AccountId,
settlement: &Asset,
) -> Pnl {
let key = (account, settlement.clone());
self.realized
.with(&key, |entry| *entry)
.unwrap_or(Pnl::ZERO)
}
}
```
The built-in policy types are `NoLocking`, `FullLocking`, and
`IndexLocking`. Access is closure-based: `with` provides read-only
scoped access, `with_mut` provides read/write scoped access with
on-demand insertion, and `remove` deletes an entry. The access scope ends when
the closure returns. `Storage` is intentionally not `Clone` - share it through
an explicit owner type or borrow `&Storage<...>`.
## Example: Engine-Owned Builder Use
Rust
```rust
use openpit::Engine;
let builder = Engine::builder::<(), (), ()>().full_sync();
let counters = builder.storage_builder().create_for_bound_key::<&'static str, u64>();
counters.with_mut("ticks", || 0, |value, _is_new| {
*value += 1;
});
assert_eq!(counters.with(&"ticks", |value| *value), Some(1));
assert!(counters.remove(&"ticks"));
```
## Choosing a policy
- **Caller does not drive the engine from multiple threads** -> no-sync.
Cheapest mode, no synchronization overhead.
- **Shared engine, no scheduling guarantees from the host** -> full-sync.
Safe default.
- **Shared engine, each account is pinned to a single chain (queue or
worker)** -> account sync. Trades a correctness obligation for unbounded
per-account parallelism.
- **Anything else** -> custom policy passed through the engine builder.
## Related Pages
- [Threading Contract](Threading-Contract.md): Per-call execution model
the storage operates under
- [Policies](Policies.md): Built-in policies that already use storage
internally
- [Policy API](Policy-API.md): Language-specific interfaces for custom
policies