refactor: enforce tx input encryption on submission - #2381
Conversation
igamigo
left a comment
There was a problem hiding this comment.
Leaving a couple of comments for now but the overall shape looks good
| pub async fn create_genesis_aware_rpc_client_with_commitment( | ||
| rpc_url: &Url, | ||
| timeout: Duration, | ||
| ) -> Result<(RpcClient, Word)> { |
There was a problem hiding this comment.
Should this not just replace the previous function?
| let (pool, genesis_commitment) = | ||
| create_genesis_aware_rpc_client_pool(&rpc_url, Duration::from_secs(30), connections) | ||
| .await | ||
| .expect("failed to create RPC client pool"); |
There was a problem hiding this comment.
Instead of the genesis commitment, I believe this could also just return the TransactionInputsSealer
| /// Holding one avoids re-fetching the key per submission; callers should discard it when the | ||
| /// validator reports an unknown key id, which is how a key change is detected. | ||
| #[derive(Debug, Clone)] | ||
| pub struct TransactionInputSealer { |
There was a problem hiding this comment.
nit: I'd name this TransactionInputsSealer instead
| scheme: u32, | ||
| key_id: Vec<u8>, | ||
| sealing_key: SealingKey, |
There was a problem hiding this comment.
Wonder if this should be its own domain type and then we could have an impl From<proto::transaction::TransactionEncryptionKey> for TransactionEncryptionKey
| impl TransactionInputSealer { | ||
| /// Builds a sealer from a served encryption key and the genesis commitment of the network the | ||
| /// inputs will be submitted to. | ||
| pub fn new( |
There was a problem hiding this comment.
Should we use/verify attestation here?
| async fn call_submit_proven_transaction( | ||
| &self, | ||
| tx: &ProvenTransaction, | ||
| sealed: Option<proto::transaction::SealedTransactionInputs>, |
There was a problem hiding this comment.
Are we using the None-option somewhere? Now that it is enforced, could we ever need an Option here?
| let original_accept_header = metadata.get(http::header::ACCEPT.as_str()).cloned(); | ||
|
|
||
| tracing::trace!(target: LOG_TARGET, ?request); | ||
| tracing::trace!(target: LOG_TARGET, "Received transaction submission"); |
There was a problem hiding this comment.
Should we add any extra information herE?
| let result = self.inner.clone().submit_proven_tx(request).await; | ||
|
|
||
| // A validator restarted with a different encryption key rejects with `failed_precondition`. | ||
| // Drop the cached key so the next attempt seals against the current one. | ||
| if let Err(status) = &result | ||
| && status.code() == tonic::Code::FailedPrecondition | ||
| { | ||
| self.invalidate_sealer().await; | ||
| } | ||
| result?; | ||
|
|
||
| Ok(()) |
There was a problem hiding this comment.
| let result = self.inner.clone().submit_proven_tx(request).await; | |
| // A validator restarted with a different encryption key rejects with `failed_precondition`. | |
| // Drop the cached key so the next attempt seals against the current one. | |
| if let Err(status) = &result | |
| && status.code() == tonic::Code::FailedPrecondition | |
| { | |
| self.invalidate_sealer().await; | |
| } | |
| result?; | |
| Ok(()) | |
| match self.inner.clone().submit_proven_tx(request).await { | |
| Ok(_) => Ok(()), | |
| Err(status) => { | |
| // A validator restarted with a different encryption key rejects with | |
| // `failed_precondition`. Drop the cached key so the next attempt seals against the | |
| // current one. | |
| if status.code() == tonic::Code::FailedPrecondition { | |
| self.invalidate_sealer().await; | |
| } | |
| Err(status) | |
| }, | |
| } |
huitseeker
left a comment
There was a problem hiding this comment.
Main item: let's check the attestation in TransactionInputSealer::new.
|
|
||
| let mut cache = self.encryption_key_cache.lock().await; | ||
| if let Some(cached) = cache.as_ref() | ||
| && cached.fetched_at.elapsed() < ENCRYPTION_KEY_CACHE_TTL |
There was a problem hiding this comment.
Could we drop this cache, or invalidate it when a forwarded submission returns FailedPrecondition? After a validator restarts with a new key, the submit response tells the client to re-fetch, but this branch serves the same stale key for up to 30 seconds. The ntx builder then seals with the old key again and burns another note attempt instead of recovering. A restart test that changes the upstream key would cover this path.
There was a problem hiding this comment.
Shouldn't these keys be very long lived, relatively speaking?
I imagine the clients will already be caching the key themselves, so imo we can simplify and just always forward the request.
| // A validator restarted with a different encryption key rejects with `failed_precondition`. | ||
| // Drop the cached key so the next attempt seals against the current one. | ||
| if let Err(status) = &result | ||
| && status.code() == tonic::Code::FailedPrecondition |
There was a problem hiding this comment.
Could this retry once after invalidating the sealer? NtxContext::submit only retries statuses accepted by is_transient_status, and that function excludes FailedPrecondition. This error therefore escapes immediately, so the actor marks every note in the candidate failed before a later block can try the fresh key. Re-fetching and resealing once here would make the key-change path recover without penalizing the notes.
| impl TransactionInputSealer { | ||
| /// Builds a sealer from a served encryption key and the genesis commitment of the network the | ||
| /// inputs will be submitted to. | ||
| pub fn new( |
| [Service] | ||
| Type=exec | ||
| Environment="OTEL_SERVICE_NAME=miden-validator" | ||
| EnvironmentFile=-/etc/opt/miden-validator/env |
There was a problem hiding this comment.
Why is this added?
| // Opaque identifier of the encryption key the inputs were sealed against, copied from | ||
| // [TransactionEncryptionKey.key_id]. | ||
| bytes key_id = 1; | ||
|
|
||
| // The transaction inputs sealed with [miden_protocol::crypto::ies::SealingKey], encoded using | ||
| // the [miden_protocol::crypto::utils::Serializable] implementation for | ||
| // [miden_protocol::crypto::ies::SealedMessage]. | ||
| bytes ciphertext = 2; |
There was a problem hiding this comment.
Question - will we only ever support a single key type? Or should this be an enum?
| nodes without one forward it to the upstream RPC source. Responses are cached briefly, since every submitting client | ||
| must fetch the key first and a validator's answer does not change while it is running. |
There was a problem hiding this comment.
Should keys not have an expiry set? At the moment this expiry is simply forever, so I don't know we would cache things briefly only. I'll revisit this when I see the impl.
| let original_accept_header = metadata.get(http::header::ACCEPT.as_str()).cloned(); | ||
|
|
||
| tracing::trace!(target: LOG_TARGET, ?request); | ||
| tracing::trace!(target: LOG_TARGET, batch_size = request.sealed_transaction_inputs.len()); |
There was a problem hiding this comment.
| tracing::trace!(target: LOG_TARGET, batch_size = request.sealed_transaction_inputs.len()); | |
| tracing::trace!(target: LOG_TARGET, batch.size = request.sealed_transaction_inputs.len()); |
|
|
||
| let mut cache = self.encryption_key_cache.lock().await; | ||
| if let Some(cached) = cache.as_ref() | ||
| && cached.fetched_at.elapsed() < ENCRYPTION_KEY_CACHE_TTL |
There was a problem hiding this comment.
Shouldn't these keys be very long lived, relatively speaking?
I imagine the clients will already be caching the key themselves, so imo we can simplify and just always forward the request.
| /// Genesis commitment of the network being submitted to, bound into the associated data of | ||
| /// sealed transaction inputs. | ||
| genesis_commitment: Word, | ||
| /// Cached sealer for transaction inputs, fetched on first submission. | ||
| sealer: Arc<RwLock<Option<TransactionInputSealer>>>, |
There was a problem hiding this comment.
Same here. There is no reason this can't already be populated, or handled by the client.
| spawn_blocking_in_current_span(move || { | ||
| UnsealingKey::X25519XChaCha20Poly1305(secret_key) | ||
| .unseal_bytes_with_associated_data(message, &associated_data) | ||
| .context("AEAD authentication failed") | ||
| }) | ||
| .await |
There was a problem hiding this comment.
Why are these functions async?
| scheme: Self::scheme_id(), | ||
| key_id: self.key_id(), |
There was a problem hiding this comment.
Should these not be an enum? We surely don't support an open-ended set of schemas?
| pub struct TransactionInputSealer { | ||
| scheme: u32, | ||
| key_id: Vec<u8>, | ||
| sealing_key: SealingKey, | ||
| genesis_commitment: Word, | ||
| } |
There was a problem hiding this comment.
Are we sure this shouldn't be an enum?
| pub fn transaction_inputs_associated_data( | ||
| scheme: u32, | ||
| key_id: &[u8], | ||
| genesis_commitment: Word, | ||
| tx_id: TransactionId, | ||
| ) -> Vec<u8> { |
There was a problem hiding this comment.
Should this be pub? Where else is it used? I assume that's why it isn't:
| pub fn transaction_inputs_associated_data( | |
| scheme: u32, | |
| key_id: &[u8], | |
| genesis_commitment: Word, | |
| tx_id: TransactionId, | |
| ) -> Vec<u8> { | |
| impl TransactionInputSealer { | |
| fn build_associated_data(&self, tx_id: TransactionId) -> Vec<u8> { | |
| ... | |
| } | |
| } | |
|
Closing, superseeded by #2389 |
Summary
This PR makes sealing mandatory. New flow now implies clients calling
GetTransactionEncryptionKeybefore submission to seal inputs.Plain
transaction_inputsis gone from both submission messages, replaced by:Changelog