Skip to main content

Technical Spec

EDITED2024-10-27
StatusDraft

Xythum Darkpool Gateway: Core Protocol Specification

Abstract

This document specifies the Xythum Privacy Gateway, a foundational, protocol-agnostic privacy layer for EVM-compatible blockchains. The gateway is designed to enable confidential, anonymous interaction with public, transparent DeFi applications without requiring any modification to the target protocols. It achieves this through a novel architecture combining a ZK-SNARK powered dark pool (TokenPool), an on-chain smart contract proxy (PrivacyProxy), and a set of decentralized off-chain services for state management and data availability. Users onboard assets into the private pool, creating shielded commitments, and then authorize the PrivacyProxy to execute transactions on their behalf using ECDSA signatures derived from a non-custodial, wallet-derived secret. This specification details the system architecture, core components, on-chain mechanics, cryptographic methods, and the complete user lifecycle.

Motivation​

The core value proposition of decentralized finance is its radical transparency. However, this feature has become a critical impediment to its evolution, creating a "crisis of confidentiality" that stifles both institutional adoption and retail security.

  • Institutional & Sophisticated Traders: The public nature of blockchains makes professional trading strategies untenable. Actions such as accumulating a position, managing vesting schedules, or rebalancing a large portfolio are broadcast to the entire market in real-time. This "strategic leakage" invites predatory MEV exploitation (front-running, sandwich attacks) and allows competitors to copy strategies, severely eroding alpha and operational efficiency. Traditional finance relies on dark pools and confidential execution venues for this reason, a capability largely absent in DeFi today.
  • Retail & Mass Adoption: For the average user, the lack of privacy is a severe security and social risk. Every transaction, no matter how small, can be traced back to a wallet address, potentially exposing the user's entire financial history, net worth, and spending habits. This concept of "financial self-doxxing" is a major deterrent to mainstream adoption, as users rightly expect a degree of privacy in their financial affairs, analogous to a traditional bank account.
  • Protocols, DAOs, & Web3 Businesses: The transparent ledger complicates treasury management, making strategic diversification or payroll operations public knowledge. It prevents the creation of novel applications that require confidentiality, such as private on-chain voting or MEV-resistant trading features offered natively by a DEX.

The Xythum Privacy Gateway is motivated by the need to solve these problems. It aims to provide a universal, opt-in privacy layer that can be overlaid on any existing DeFi protocol, thereby unlocking the next wave of capital, users, and innovation without compromising the composability of the underlying ecosystem.

Introduction​

The Xythum Gateway is an infrastructure-level solution that decouples a user's on-chain identity (their EOA) from their on-chain actions. It achieves this by introducing a system where a user's funds are held in a private, ZK-shielded state, and all external interactions are routed through a common, anonymized smart contract proxy.

A user's journey through the Xythum ecosystem follows a distinct lifecycle:

  1. Onboarding: The user generates a unique, non-custodial "secret key" by signing a static message with their primary EOA wallet. This secret becomes the root of their private identity.
  2. Shielding: The user deposits assets into the TokenPoolV2 contract, creating a private "commitment" (a UTXO-like note) in a Merkle tree.
  3. Interaction: The user funds the PrivacyProxy from their shielded balance by generating a ZK proof. They can then interact with any integrated DeFi protocol (e.g., a DEX, lending protocol, etc.) by signing authorization messages with their secret key. The PrivacyProxy executes these transactions, appearing as the sole msg.sender to the target protocol.
  4. Off-boarding: The user withdraws their funds from the PrivacyProxy back into the TokenPoolV2 as a new private note, and can subsequently withdraw from the pool back to any EOA of their choosing.

This entire process is supported by a set of off-chain services that provide a seamless user experience without sacrificing decentralization or security.

System Design & Architecture

System Design

On-Chain Layer (Solidity Contracts)​

This layer represents the trust-minimized foundation of the protocol.

  • TokenPoolV2.sol: The core dark pool contract. It manages the Merkle tree of user commitments, holds the shielded assets, and verifies ZK proofs for all private actions (withdrawals, transfers, claims).
  • PrivacyProxy.sol: The public-facing gateway. It holds collateral on behalf of all private users and is the msg.sender for all interactions with external protocols. It enforces ownership and authorizes actions via ECDSA signature verification.
  • ZK Verifiers: A set of immutable smart contracts (WithdrawTransferHonkVerifier.sol, ClaimHonkVerifier.sol) that verify the UltraPLONK proofs generated by the client.
  • Target Protocols: Any standard, permissionless DeFi protocol. The system is demonstrated with ClearingHouseV2.sol.

Off-Chain Services Layer (Rust)​

This layer provides the necessary data and indexing for a usable client experience, without holding any user funds or secrets.

  • The Watchtower (Indexer): A robust Rust service that subscribes to all relevant on-chain events from the TokenPoolV2, PrivacyProxy, and integrated ClearingHouseV2 contracts. It parses these events and populates a persistent database with a queryable representation of the system's state (e.g., lists of open positions, unspent notes).
  • The Data Availability (DA) Layer (Server): A secure Rust API server that acts as the primary interface for the frontend. It serves the data indexed by the Watchtower and manages the storage of user-specific encrypted metadata. All user-specific queries are authenticated via signatures from the user's secret key.

Client-Side Layer (TypeScript & ZK Libraries)​

All sensitive operations occur exclusively on the client-side, ensuring the protocol is fully non-custodial.

  • Secret Management: The user's secret key is derived and managed entirely within the browser's memory. It is never transmitted or stored on any server.
  • State Management: The client fetches its indexed state (e.g., list of position IDs, unspent notes) from the DA Layer. It also fetches and stores its own encrypted metadata blob (containing the latest nullifier nonce and commitment info).
  • ZK Proof Generation: The client's browser uses @noir-lang/noir_js and @aztec/bb.js to compile ZK circuits into WebAssembly and generate UltraPLONK proofs for all private TokenPoolV2 interactions. This process is computationally intensive and happens entirely on the user's machine.

Core Components & On-Chain Mechanics

Cryptographic Primitives​

Secret & Public Key Derivation​

A user's identity within the private system is rooted in a deterministic, non-custodial secret key. This key is the foundation for all private actions, including commitment creation, signature generation, and note claiming.

  • Secret Key (secret_key): A bytes32 private key. It is derived client-side by having the user sign a static, domain-specific message with their primary EOA wallet. This process is both deterministic and non-custodial, as the secret can be regenerated at any time on any device with access to the user's EOA, but is never stored on a server.
const message = "Xythum Darkpool Gateway Secret v1";
const signature = await eoaWallet.signMessage(message);
const secret_key = ethers.keccak256(signature);

const secretWallet = new ethers.Wallet(secret_key);
const ownerPubKey = ethers.solidityPackedKeccak256(["address"], [secretWallet.address]);

Commitments, Nullifiers, and Notes​

The TokenPoolV2 operates on a UTXO-like model where user balances are represented as private commitments.

  • Commitment (commitment): A bytes32 hash stored as a leaf in the Merkle tree. It represents a specific value of a specific asset owned by the user. Its construction ensures that it cannot be linked to the user's EOA.​

fn commitment_hasher(nullifier: Field, value: Field, label: Field, secret: Field) -> Field {
    let precommitment = poseidon2_hash([nullifier, secret]);
    let commitment = poseidon2_hash([value, label, precommitment]);
    commitment
}
  • Where label is the address of the asset (e.g., USDC token contract).

Nullifier (nullifier): A bytes32 value that is revealed when a commitment is spent. Each commitment has a unique nullifier. The TokenPoolV2 contract maintains a mapping of spent nullifiers to prevent double-spending. Nullifiers must be deterministic so the user can regenerate them, but appear random to the public.

  • Generation Logic: The nullifier is derived from the user's secret_key and a strictly increasing nonce, managed by the off-chain DA Layer.

Notes: A mechanism for transferring value privately between users (or to oneself). A Note is an on-chain record created during a transfer or withdrawCollateralToDarkPool operation. It contains an amount and a receiverHash, but is not yet a spendable commitment.

  • Receiver Hash: A bytes32 hash derived from the recipient's "receiver secret." This allows the intended recipient to prove they are the owner of the note when they claim it.

ZK-SNARKs (UltraPLONK)​

The protocol utilizes the UltraPLONK proof system, a highly efficient and universal ZK-SNARK construction. All proofs are generated client-side using circuits written in the Noir language and compiled to WebAssembly for in-browser execution.

  • WithdrawTransferCircuit: Used for withdraw, transfer, and approveWithdrawal functions. It proves that the user knows a secret and nullifier corresponding to a valid commitment within a known Merkle root, and that the output values are consistent with the input values.
  • ClaimCircuit: Used for the claim function. It proves that the user knows the receiver_secret that hashes to the receiverHash of an unspent note. It can optionally also prove ownership of an existing commitment to merge the note's value with it.

On-Chain Components

Darkpool.sol

The darkpool contract is the core privacy component. It is a multi-asset, UTXO-based dark pool.

  • State:
    • MerkleTreeLib.Tree tree: The on-chain Merkle tree storing all commitments.
    • mapping(uint256 => bytes32) roots: A rolling history of the last 100 Merkle roots to allow proofs to be generated against a recent, valid state.
    • mapping(bytes32 => bool) isNullifierSpent: A mapping to prevent double-spends.
    • mapping(bytes32 => Note) notes: A mapping of noteId to Note structs for unspent notes.
    • IVerifier withdrawVerifier, IVerifier claimVerifier: Addresses of the on-chain verifier contracts.
  • Key Functions:
    • deposit(amount, precommitment): Public function for a user's EOA to deposit funds and create their first commitment in the tree.
    • approveWithdrawal(proofParams): A private action. Validates a ZK proof, spends the user's old commitment, creates a new commitment for the remaining balance, and approves the msg.sender (the PrivacyProxy) to withdraw the specified amount.
    • depositFor(receiverHash, amount): A public function allowing any EOA or contract (like the PrivacyProxy) to deposit funds and create a Note for a specified receiverHash.
    • claim(proofParams): A private action. Validates a ZK proof of note ownership and merges the note's value into the user's existing commitment (or creates a new one).
    • withdraw(receiver, proofParams): A private action. Validates a ZK proof, spends the user's commitment, creates a new one for the remaining balance, and transfers the withdrawn amount to a public receiver address.
    • transfer(proofParams, receiverHash): A private action. Validates a ZK proof, spends the user's commitment, creates a new one for the remaining balance, and creates a new Note for the specified receiverHash.

The PrivacyProxy is the universal interaction layer. It is stateless regarding user balances but stateful regarding position ownership.

While the on-chain contracts provide the foundation for trust-minimized execution, the off-chain services are critical for providing a usable, efficient, and scalable user experience. These services are designed to be non-custodial and trust-minimized; they manage queryable data and user metadata but never have access to user secrets or funds.

The Watchtower (Indexer Service)​

The Watchtower is a high-performance Rust service responsible for creating a queryable representation of on-chain state.

  • Objective: To listen to all relevant smart contract events in real-time and populate a persistent database. This transforms the blockchain's append-only log into an indexed, efficient database that can be queried by the API layer.
  • Core Functionality:
    1. Event Subscription: The Watchtower maintains a persistent WebSocket connection to an EVM node and subscribes to event streams from multiple contracts simultaneously.
    2. Historical Sync: On startup, the Watchtower queries all historical logs in batched chunks from a specified genesis block to the current block, ensuring a complete state reconstruction.
    3. Real-Time Indexing: After the historical sync, it transitions to listening for new events on the latest blocks.

The Data Availability (DA) Layer (API Server)​

The DA Layer is a secure, authenticated Rust API server ork. It is the sole entry point for the frontend client.

  • Objective: To serve data indexed by the Watchtower and to manage user-specific encrypted metadata.
  • Authentication: All endpoints that access private user data (/private/*) are protected. The user's client must provide an X-Signature header containing an ECDSA signature of an X-Message header. The signature must be verifiable against the user's ownerPubKey. This proves ownership of the secret key without revealing it.

User Lifecycle & Interaction Flow​

This section details the step-by-step journey of a user ("Triffiny") interacting with the entire system, from onboarding to exiting.

Security Guarantees & Threat Model​

The protocol is designed to be secure under the assumption that the underlying blockchain (e.g., BNB Chain) is secure and that the cryptographic primitives (UltraPLONK, Keccak256, ECDSA) are sound.

  • Non-Custodial Design: At no point do the Xythum developers, off-chain service operators, or any other third party have the ability to move, freeze, or censor a user's funds. All private state is controlled exclusively by the user's secret_key, which is never transmitted.
  • Double-Spend Prevention: The TokenPoolV2 contract's isNullifierSpent mapping is the canonical source of truth for spent commitments. A user cannot spend the same private UTXO twice. The strictly increasing nonce managed by the client and stored on the DA Layer prevents a user from accidentally generating a proof for an already-used nullifier.
  • ZK Proof Soundness: The soundness of the UltraPLONK proof system, verified by the on-chain verifier contracts, ensures that a user cannot create fraudulent transactions. For example, they cannot generate a valid proof to withdraw more value from a commitment than it actually contains, nor can they create a new commitment with an incorrect remaining balance.
  • Access Control: All state-changing functions in the PrivacyProxy related to a specific position are protected by ECDSA signature verification. An attacker who compromises a user's EOA wallet cannot access their private trading account without also compromising their derived secret_key. This provides an additional layer of security.
  • Off-Chain Service Security: The Watchtower and DA Layer are considered "trusted" for data availability and convenience, but not for security.
    • Threat: A malicious or compromised DA Layer operator could provide a user with a stale Merkle root or incorrect metadata.
    • Mitigation: This would cause the user's client-side proof generation to fail, or the subsequent on-chain transaction to revert. The user would not lose funds, but their ability to transact would be temporarily halted. A robust frontend client should have the ability to fall back to a public RPC node to fetch critical data (like the latest Merkle root) to verify the DA Layer's integrity. The system is designed to fail safely.

The Community-Enforced Safe Harbor: A Decentralized Compliance Framework​

Xythum's core mission is to provide commercial and personal privacy, not to facilitate illicit activity. We recognize that for a privacy protocol to thrive in the long term, it must have a credible and transparent mechanism for addressing legitimate law enforcement concerns without compromising the privacy of the vast majority of law-abiding users.

The "Community-Enforced Safe Harbor" is a long-term vision for a decentralized compliance framework.

  • The Problem with Centralized Compliance: Traditional privacy protocols with centralized compliance mechanisms (where a single company holds "view keys") re-introduce a single point of failure and trust, defeating the purpose of decentralization.
  • The Xythum Solution: Threshold Decryption:
    1. Encrypted Metadata: The DA Layer stores user metadata (nonce, commitment info) as an encrypted blob. The encryption key is derived from the user's secret_key.
    2. The "Master Decryption Key": A theoretical "master" decryption key, capable of decrypting any user's metadata blob on the DA Layer, will be generated. This key will not be held by any single entity, including the Xythum team.
    3. Key Sharding: This master key will be split into multiple shards using a threshold decryption scheme (e.g., Shamir's Secret Sharing). For instance, a (7 of 12) scheme could be created, where any 7 of the 12 total shards are required to reconstruct the key.
    4. Decentralized Key Holders: These shards will be distributed to a council of highly reputable, geographically and politically diverse, independent entities. The composition of this "Safe Harbor Council" would be determined by the Xythum DAO. Potential members could include:
      • Major figures in the blockchain community (e.g., foundation members).
      • Reputable blockchain security and auditing firms.
      • Established legal firms specializing in digital assets.
      • In the future, willing and technically capable regulatory bodies (e.g., a specific department within the DOJ or Interpol).
  • The Workflow for Legitimate Law Enforcement Requests:
    1. Official Request: A legitimate law enforcement agency presents a valid, legally-binding warrant or court order to the Xythum DAO concerning a specific set of illicit transactions.
    2. DAO Governance Vote: The DAO votes on whether the request is valid and meets the high standards for a privacy override (e.g., confirmed exploit from a major protocol, funds traced directly to a sanctioned address).
    3. Quorum Activation: If the vote passes, the DAO formally requests a quorum of the Safe Harbor Council members (e.g., 7 of them) to participate in a decryption ceremony.
    4. Targeted Decryption: The council members use their key shards in a secure multi-party computation (MPC) to decrypt only the specific metadata blobs associated with the illicit funds. This would reveal the nullifier nonces and commitment history of the malicious actor, allowing for their on-chain activity within the gateway to be deanonymized.
  • Guarantees:
    • No Unilateral Decryption: No single entity, not even the Xythum team, can decrypt user data.
    • High Bar for Action: The process requires both a public DAO vote and the active cooperation of a supermajority of trusted, independent key holders, preventing frivolous or malicious requests.
    • Surgical Precision: The privacy of all other users remains completely unaffected.
sequenceDiagram
    participant Triffiny
    participant ClientUI as 🖥️ Frontend Client
    participant DALayer as 📡 DA Layer (API)
    participant TokenPool as 📄 TokenPoolV2
    participant PrivacyProxy as 🔗 PrivacyProxy

    Note over Triffiny, ClientUI: **Phase 1: Onboarding**
    Triffiny->>ClientUI: Connects EOA Wallet
    ClientUI->>Triffiny: Prompts to sign message "Xythum... Secret v1"
    Triffiny->>ClientUI: Signs message
    ClientUI->>ClientUI: Derives `secret_key` and `ownerPubKey`

    Note over Triffiny, ClientUI: **Phase 2: Initial Shielded Deposit**
    ClientUI->>DALayer: GET /private/metadata (auth with `ownerPubKey`)
    DALayer-->>ClientUI: 404 Not Found (or empty metadata)
    ClientUI->>ClientUI: Initializes metadata (nonce = 0)
    ClientUI->>ClientUI: Calculates `nullifier` for deposit (using nonce=1)
    Triffiny->>+ClientUI: Inputs deposit amount (e.g., 10,000 USDC)
    ClientUI->>TokenPool: approve(TokenPool, 10000)
    ClientUI->>TokenPool: deposit(10000, precommitment)
    TokenPool-->>ClientUI: TxReceipt with `CommitmentInserted` event
    ClientUI->>ClientUI: Parses `leafIndex` from event
    ClientUI->>ClientUI: Updates metadata (nonce=1, commitment_info={value, leafIndex})
    ClientUI->>DALayer: POST /private/metadata (sends encrypted metadata)
    DALayer-->>ClientUI: 200 OK

    Note over Triffiny, ClientUI: **Phase 3: Funding the Trading Account**
    Triffiny->>+ClientUI: Decides to deposit 2000 USDC into perps
    ClientUI->>DALayer: GET /private/metadata (gets latest nonce & commitment)
    DALayer-->>ClientUI: Returns encrypted metadata
    ClientUI->>ClientUI: Decrypts metadata
    ClientUI->>ClientUI: Generates ZK proof for `approveWithdrawal`
    ClientUI->>PrivacyProxy: depositCollateralFromDarkPool(proof)
    PrivacyProxy->>TokenPool: approveWithdrawal(proof)
    TokenPool-->>PrivacyProxy: approve(Proxy, 2000)
    PrivacyProxy->>PrivacyProxy: transferFrom(TokenPool, 2000)
    PrivacyProxy-->>ClientUI: TxReceipt
    ClientUI->>ClientUI: Parses new `leafIndex` for remaining balance
    ClientUI->>ClientUI: Updates metadata (nonce=3, new commitment_info)
    ClientUI->>DALayer: POST /private/metadata (updates metadata)
    DALayer-->>ClientUI: 200 OK

    Note over Triffiny, ClientUI: **Phase 4: Trading**
    Triffiny->>+ClientUI: Opens a 10x Long position
    ClientUI->>ClientUI: Signs "OPEN_POSITION" message with `secret_key`
    ClientUI->>PrivacyProxy: openPosition(pubKey, positionId, margin, signature)
    
    Note over Triffiny, ClientUI: **Phase 5: Exiting**
    Triffiny->>+ClientUI: Closes position, then withdraws from proxy
    ClientUI->>ClientUI: Signs "CLOSE_POSITION", then "WITHDRAW_COLLATERAL"
    ClientUI->>PrivacyProxy: closePosition(...) -> withdrawCollateralToDarkPool(...)
    PrivacyProxy->>TokenPool: depositFor(...) -> creates a new note for Triffiny
    
    Triffiny->>+ClientUI: Claims the note
    ClientUI->>DALayer: GET /private/notes/unspent
    DALayer-->>ClientUI: Returns unspent notes
    ClientUI->>ClientUI: Generates ZK proof for `claim`
    ClientUI->>TokenPool: claim(proof)
    
    Triffiny->>+ClientUI: Withdraws final balance to EOA
    ClientUI->>ClientUI: Generates ZK proof for `withdraw`
    ClientUI->>TokenPool: withdraw(proof)
    TokenPool-->>Triffiny: Final funds sent to EOA

  • PRE DEVELOPMENT SPEC

https://hackmd.io/vKVtIvVBQlG6YLX8ZmgRZA