Messaging & Encrypted Data Subsystem

The Shamwari Messaging & Encrypted Data Subsystem (nxt.messaging) provides a protocol-native, decentralized framework for on-chain communications, arbitrary data storage, and post-quantum encrypted messaging built directly into the Shamwari core ledger engine.

By handling messaging and encrypted payload attachments directly within nxt.messaging as transaction appendices—rather than relying on off-chain side channels, centralized messaging servers, or smart contract storage calls—Shamwari delivers zero-counterparty risk, quantum-resistant end-to-end encryption (E2EE), tamper-proof audit trails, and automatic data retention across BetaChains.

---

Core Capabilities & Architecture

• Protocol-Native On-Chain Messaging: Messages are embedded directly into transaction instances as transaction appendices (Message, EncryptedMessage, EncryptToSelfMessage), removing external dependencies and smart contract overhead.
• Post-Quantum Key Encapsulation Mechanism (KEM): Integrates lattice-based post-quantum cryptography (CRYSTALS-Kyber via KyberCipherText and KyberParams) to protect message content against quantum decryption attacks.
• Dual Recipient Encryption Models: Supports both recipient-targeted messaging (EncryptedMessage) and sender self-archiving (EncryptToSelfMessage), allowing nodes and users to store encrypted private ledger notes and documents on-chain.
• Compressed & Text-Flagged Payload Layouts: Encrypted payloads include bitmask flags (isText, isCompressed) and stream-compression mechanics to optimize storage efficiency on BetaChain ledgers.
• Size-Based Dynamic Fee Structure: Enforces a baseline byte-sized fee schedule (ENCRYPTED_MESSAGE_FEE) linked to payload length to prevent ledger spam and memory pool pollution.
• Seamless Attachment Compatibility: Messaging appendices can be attached to standard transfers, payment transactions, order executions, or submitted as standalone messaging transactions.

---

Encrypted Message Appendices & Structure

The subsystem defines three primary appendix types in nxt.messaging:

Appendix Type	Class Name	Functional Purpose
Plain Text Message	Message	Unencrypted, public text or binary payload readable by all network participants.
Encrypted Message	EncryptedMessage	End-to-end encrypted payload targetable to a specific recipient account using post-quantum Kyber KEM.
Encrypt To Self Message	EncryptToSelfMessage	Encrypted payload targeted exclusively to the sender account for private note-taking, document storage, or off-chain data sync.

Binary Attachment Wire Format

The serialized byte layout of an encrypted message appendix (AbstractEncryptedMessageAppendix) is structured as follows:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|     Flags     |          Data Length          |               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                        Encrypted Data                         |
|                             ...                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                    Kyber CipherText Vector                    |
|                             ...                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Field Specifications

1. Flags (1 byte): Bitmask denoting message characteristics:
   • Bit 0 (0x01): isText — 1 if payload is UTF-8 text; 0 if raw binary data.
   • Bit 1 (0x02): isCompressed — 1 if payload was compressed prior to encryption.
2. Data Length (2 bytes): Unsigned 16-bit integer representing the byte length of the symmetric encrypted ciphertext payload.
3. Encrypted Data (N bytes): Symmetrically encrypted payload bytes derived from the shared secret.
4. Kyber CipherText (C bytes): Encapsulated key vector generated by the post-quantum Kyber KEM (KyberCipherText.getC()).

---

Post-Quantum Cryptographic Protocol Architecture

Shamwari messaging leverages a hybrid Key Encapsulation Mechanism (KEM) based on CRYSTALS-Kyber:

 Sender                                                             Recipient
┌──────────────────────────────────────┐                   ┌──────────────────────────────────────┐
│  Recipient Public Key (KyberParams)  │                   │ Recipient Private Key (KyberParams)  │
└──────────────────┬───────────────────┘                   └──────────────────┬───────────────────┘
                   │                                                          │
                   ▼                                                          │
┌──────────────────────────────────────┐                                      │
│  Kyber.encapsulate(recipientPubKey)  │                                      │
├──────────────────┬───────────────────┤                                      │
│ Shared Secret S  │ KyberCipherText C │                                      │
└────────┬─────────┴─────────┬─────────┘                                      │
         │                   │                                                │
         ▼                   │ (Published on BetaChain Ledger)                │
┌──────────────────┐         │                                                │
│ AES-GCM Encrypt  │         │                                                │
│  Plaintext Data  │         │                                                │
└────────┬─────────┘         │                                                │
         │                   │                                                │
         ▼                   │                                                │
┌──────────────────┐         │                                                │
│ Encrypted Data   │         │                                                │
└────────┬─────────┘         │                                                │
         │                   │                                                │
         └─────────┬─────────┘                                                │
                   │                                                          │
                   ▼                                                          ▼
    AbstractEncryptedMessageAppendix ───────────────────► ┌──────────────────────────────────┐
                                                             │ Kyber.decapsulate(C, privKey)    │
                                                             ├──────────────────────────────────┤
                                                             │ Derives Shared Secret S          │
                                                             ├──────────────────────────────────┤
                                                             │ AES-GCM Decrypt Encrypted Data   │
                                                             └──────────────────────────────────┘

Encryption & Key Agreement Steps

1. Key Encapsulation: The sender fetches the recipient's post-quantum public key (KyberPublicKey) from public.public_key and executes Kyber encapsulation: $(S, C) = \text{Kyber.encapsulate}(PK_{\text{recipient}})$ Where $S$ is the derived 256-bit symmetric key and $C$ is the serialized KyberCipherText.
2. Payload Compression & Encryption: If enabled, the message string or binary payload is compressed and then encrypted using AES-GCM initialized with key $S$: $\text{EncryptedData} = \text{AES-GCM-Encrypt}_{S}(\text{Compress}(\text{Payload}))$
3. Appendix Construction: The resulting EncryptedData object and KyberCipherText $C$ are packed into an EncryptedMessageAppendix and attached to the transaction.
4. Decryption: The recipient retrieves $C$ from the ledger, decapsulates $S = \text{Kyber.decapsulate}(C, SK_{\text{recipient}})$, and decrypts the payload.

---

Mathematical Formulas & Dynamic Baseline Fees

To preserve network capacity and eliminate unpriced storage inflation across BetaChains, AbstractEncryptedMessageAppendix calculates transaction fees dynamically using a size-based fee model (ENCRYPTED_MESSAGE_FEE).

1. Appendix Size Calculation Formula

The total byte footprint ($\text{Size}_{\text{appendix}}$) of an encrypted message appendix is derived from its header bytes and payload components:

$\text{Size}_{\text{appendix}} = 1 + 2 + \text{EncryptedData.getSize}()$

Where:

• $1 \text{ byte}$ represents the bitmask flags (isText, isCompressed).
• $2 \text{ bytes}$ represent the 16-bit payload length short integer.
• $\text{EncryptedData.getSize}() = |\text{Data}| + |\text{KyberCipherText.getC}()|$.

2. Size-Based Dynamic Fee Formula

The baseline transaction fee ($\text{Fee}_{\text{encrypted\_message}}$) scales monotonically with the payload length:

$\text{Fee}_{\text{encrypted\_message}} = \text{Fee.SizeBasedFee}\left(\frac{\text{ONE\_FXT}}{10}, \frac{\text{ONE\_FXT}}{10}, 32\right)$

Mathematically, for an effective payload byte length $L = \text{getEncryptedDataLength}() - 16$:

$\text{Fee}_{\text{encrypted\_message}} = \frac{\text{ONE\_FXT}}{10} + \max\left(0, \left\lceil \frac{L - 32}{32} \right\rceil \right) \times \frac{\text{ONE\_FXT}}{10}$

Where:

• $\text{ONE\_FXT}$ is the base network currency unit ($10^8 \text{ QNT}$).
• The initial baseline allowance is $32 \text{ bytes}$.
• Each additional $32\text{-byte}$ block adds $\frac{\text{ONE\_FXT}}{10}$ to the fee.

---

Data Model & JSON Representation

Encrypted message appendices can be represented in structured JSON format for client RPC APIs and wallet interfaces.

JSON Serialization Format

{
  "data": "a1b2c3d4e5f6...",
  "cipherText": "0f1e2d3c4b5a...",
  "isText": true,
  "isCompressed": true
}

Field	Data Type	Description
data	String (Hex)	Hexadecimal encoded representation of the encrypted payload bytes.
cipherText	String (Hex)	Hexadecimal encoded representation of the post-quantum KyberCipherText vector ($C$).
isText	Boolean	Flag indicating whether the underlying decrypted payload is plain text (true) or raw binary (false).
isCompressed	Boolean	Flag indicating whether the payload was compressed prior to encryption (true).

---

Lifecycle Event Matrix & System Hooks

The Messaging Subsystem interacts with transaction validation and processing hooks during block execution:

Stage / Component	Operation	Action & Validation
Transaction Construction	putMyBytes()	Serializes flags, data lengths, encrypted payload, and Kyber vector into binary stream.
Validation Cycle	validate()	Enforces maximum payload size limits (up to 1,000 bytes for standard attachments) and verifies Kyber vector integrity.
Fee Calculation	getBaselineFee()	Evaluates ENCRYPTED_MESSAGE_FEE based on actual byte length to calculate required FXT fee.
Block Ingestion	AFTER_BLOCK_APPLY	Embeds message metadata into ledger indexes for subscriber querying.

---

Related APIs

API	Description
SendMessage	Send message
ReadMessage	Read message
GetPrunableMessage	Get prunable message
GetPrunableMessages	List prunable messages
GetAllPrunableMessages	Get all prunable messages
VerifyPrunableMessage	Verify prunable message
DownloadPrunableMessage	Download prunable message

Primary Use Cases

• Post-Quantum P2P Messaging: Secure, quantum-resistant end-to-end encrypted direct messaging between accounts without relying on centralized servers.
• On-Chain Audit Trails & Invoicing: Attaching encrypted invoice details, purchase orders, or contract references directly to CurrencyPayment or LoanRepayment transactions.
• Private Note-Taking & Key Escrow: Utilizing EncryptToSelfMessage to archive encrypted personal notes, seed phrases, or encrypted configuration parameters directly on the ledger.
• Automated AI & Autonomous Agent Communication: Programmatic data transfers between AUTONOMOUS accounts executing algorithmic trades, multi-sig approvals, or credit scoring updates.