This is a design description of how we can go about integrating SGX with Corda. This is followed by links to our high level design for SGX infrastructure which is quite involved but independent of the possible integration points. Please feel free to comment!
For the purposes of this design we can think of SGX as a way to provision secrets to a remote node with the knowledge that only trusted code(= enclave) will operate on it. Furthermore it provides a way to durably encrypt data in a scalable way while also ensuring that the encryption key is never leaked (unless the encrypting enclave is compromised).
Broadly speaking there are two dimensions to deciding how we can integrate SGX: what we store in the ledger and where we store it.
The first dimension is the what: this relates to what we so far called the "integrity model" vs the "privacy model".
In the integrity model we rely on SGX to ensure the integrity of the ledger. Using this assumption we can cut off the transaction body and only store an SGX-backed signature over filtered transactions. Namely we would only store information required for notarisation of the current and subsequent spending transactions. This seems neat on first sight, however note that if we do this naively then if an attacker can impersonate an enclave they'll gain write access to the ledger, as the fake enclave can sign transactions as valid without having run verification.
In the privacy model we store the full transaction backchain (encrypted) and we keep provisioning it between nodes on demand, just like in the current Corda implementation. This means we only rely on SGX for the privacy aspects - if an enclave is compromised we only lose privacy, the verification cannot be eluded by providing a fake signature.
The other dimension is the where: currently in non-SGX Corda the full transaction backchain is provisioned between non- notary nodes, and is also provisioned to notaries in the case they are validating ones. With SGX+BFT notaries we have the possibility to offload the storage of the encrypted ledger (or encrypted signatures thereof) to notary nodes (or dedicated oracles) and only store bookkeeping information required for further ledger updates in non-notary nodes. The storage policy is very important, customers want control over the persistence of even encrypted data, and with the introduction of recent regulation (GDPR) unrestricted provisioning of sensitive data will be illegal by law, even when encrypted.
We'll explore the different combination of choices below. Note that we don't need to marry to any one of them, we may decide to implement several.
Privacy model + non-notary provisioning
Let's start with the model that's closest to the current Corda implementation as this is an easy segue into the possibilities with SGX. We also have a simple example and a corresponding neat diagram to indicate in a semi-handwavy way what the integration will look like.
We have a cordapp X used by node A and B. The cordapp contains a flow XFlow and a (deterministic) contract XContract. The two nodes are negotiating a transaction T2. T2 consumes a state that comes from transaction T1.
Let's assume that both A and B are happy with T2, except Node A hasn't established the validity of it yet. Our goal is to prove the validity of T2 to A without revealing the details of T1.
The following diagram shows an overview of how this can be achieved. Note that the diagram is highly oversimplified and is meant to communicate the high-level dataflow relevant to Corda.
(1) This is simplified, the actual protocol is a bit different. Namely the quote is not generated every time A requires provisioning, but is rather generated periodically.
(2) There is a way to do this check inside the enclave, however it requires switching on of the Intel ME which in general isn't available on machines in the cloud and is known to have vulnerabilities.
(3) We need symmetric trust even if the secrets seem to only flow from B to A. Node B may try to fake being an enclave to fish for information from A.
(4) The generated keys in the quotes are used to authenticate the respective parts of the DH key exchange.
(5) Sealing means encryption of data using a key unique to the enclave and CPU. The data may be subsequently unsealed (decrypted) by the enclave, even if the enclave was restarted. Also note that there is another layer of abstraction needed which we don't detail here, needed for redundancy of the encryption key.
To summarise, the journey of T1 is:
As we can see in this model each non-notary node runs their own SGX enclave and related storage. Validation of the backchain happens by secure provisioning of it between enclaves, plus subsequent verification and storage. However there is one important thing missing from the example (actually it has several, but those are mostly technical detail): the notary!
In reality we cannot establish the full validity of T2 at this point of negotiation, we need to first notarise it. This model gives us some flexibility in this regard: we can use a validating notary (also running SGX) or a non-validating one. This indicates that the enclave API should be split in two, mirroring the signature check choice in SignedTransaction.verify. Only when the transaction is fully signed and notarised should it be persisted (sealed).
This model has both advantages and disadvantages. On one hand it is the closest to what we have now - we (and users) are familiar with this model, we can fairly easily nest it into the existing codebase and it gives us flexibility with regards to notary modes. On the other hand it is a compromising answer to the regulatory problem. If we use non- validating notaries then the backchain storage is restricted to participants, however consider the following example: if we have a transaction X that parties A and B can process legally, but a later transaction Y that has X in its backchain is sent for verification to party C, then C will process and store X as well, which may be illegal.
Privacy model + notary provisioning
This model would work similarly to the previous one, except non-notary nodes wouldn't need to run SGX or care about storage of the encrypted ledger, it would all be done in notary nodes. Nodes would connect to SGX capable notary nodes, and after attestation the nodes can be sure that the notary has run verification before signing.
This fixes the choice of using validating notaries, as notaries would be the only entities capable of verification: only they have access to the full backchain inside enclaves.
Note that because we still provision the full backchain between notary members for verification, we don't necessarily need a BFT consensus on validity - if an enclave is compromised an invalid transaction will be detected at the next backchain provisioning.
This model reduces the number of responsibilities of a non-notary node, in particular it wouldn't need to provide storage for the backchain or verification, but could simply trust notary signatures. Also it wouldn't need to host SGX enclaves, only partake in the DH exchange with notary enclaves. The node's responsibilities would be reduced to the orchestration of ledger updates (flows) and related bookkeeping (vault, network map). This split would also enable us to be flexible with regards to the update orchestration: trust in the validity of the ledger would cease to depend on the transaction resolution currently embedded into flows - we could provide a from-scratch light-weight implementation of a "node" (say a mobile app) that doesn't use flows and related code at all, it just needs to be able to connect to notary enclaves to notarise, validity is taken care of by notaries.
Note that although we wouldn't require validation checks from non-notary nodes, in theory it would be safe to allow them to do so (if they want a stronger-than-BFT guarantee).
Of course this model has disadvantages too. From the regulatory point of view it is a strictly worse solution than the non-notary provisioning model: the backchain would be provisioned between notary nodes not owned by actual participants in the backchain. It also disables us from using non-validating notaries.
Integrity model + non-notary provisioning
In this model we would trust SGX-backed signatures and related attestation datastructures (quote over signature key signed by Intel) as proof of validity. When node A and B are negotiating a transaction it's enough to provision SGX signatures over the dependency hashes to one another, there's no need to provision the full backchain.
This sounds very simple and efficient, and it's even more private than the privacy model as we're only passing signatures around, not transactions. However there are a couple of issues that need addressing: If an SGX enclave is compromised a malicious node can provide a signature over an invalid transaction that checks out, and nobody will ever know about it, because the original transaction will never be verified. One way we can mitigate this is by requiring a BFT consensus signature, or perhaps a threshold signature is enough. We could decouple verification into "verifying oracles" which verify in SGX and return signatures over transaction hashes, and require a certain number of them to convince the notary to notarise and subsequent nodes to trust validity. Another issue is enclave updates. If we find a vulnerability in an enclave and update it, what happens to the already signed backchain? Historical transactions have signatures that are rooted in SGX quotes belonging to old untrusted enclave code. One option is to simply have a cutoff date before which we accept old signatures. This requires a consensus-backed timestamp on the notary signature. Another option would be to keep the old ledger around and re-verify it with the new enclaves. However if we do this we lose the benefits of the integrity model - we get back the regulatory issue, and we don't gain the performance benefits.
Integrity model + notary provisioning
This is similar to the previous model, only once again non-notary nodes wouldn't need to care about verifying or collecting proofs of validity before sending the transaction off for notarisation. All of the complexity would be hidden by notary nodes, which may use validating oracles or perhaps combine consensus over validity with consensus over spending. This model would be a very clean separation of concerns which solves the regulatory problem (almost) and is quite efficient as we don't need to keep provisioning the chain. One potential issue with regards to regulation is the tip of the ledger (the transaction being notarised) - this is sent to notaries and although it is not stored it may still be against the law to receive it and hold it in volatile memory, even inside an enclave. I'm unfamiliar with the legal details of whether this is good enough. If this is an issue, one way we could address this would be to scope the validity checks required for notarisation within legal boundaries and only require "full" consensus on the spentness check. Of course this has the downside that ledger participants outside of the regulatory boundary need to trust the BFT-SGX of the scope. I'm not sure whether it's possible to do any better, after all we can't send the transaction body outside the scope in any shape or form.
In all models we have the following actors, which may or may not overlap depending on the model:
We have two major ways of compromise:
In the case of notaries compromise means malicious signatures, for nodes it's malicious transactions, for R3 it's signing malicious enclaves, for Network Map it's signing malicious contracts, for storage it's read-write access to encrypted data, and for Intel it's forging of quotes or signing over invalid ones.
A compromise of an enclave means some form of access to the enclave's temporary identity key. This may happen through direct hardware compromise (extracting of fuse values) and subsequent forging of a quote, or leaking of secrets through weakness of the enclave-host boundary or other side-channels like Spectre(hacking). In any case it allows an adversary to impersonate an enclave and therefore to intercept enclave traffic and forge signatures.
The actors relevant to SGX are enclave hosts, storage infrastructure owners, regular nodes and R3.
Note however that even if Intel is compromised they would need to connect to the ledger participants in order to take advantage. The flow framework and the business network concept act as a form of ACL on data that would make an Intel backdoor quite useless.
As we can see we have a number of options here, all of them have advantages and disadvantages.
Privacy + non-notary
Privacy + notary
Integrity + non-notary
Integrity + notary
or privacy in case of a breach" is a business and social decision rather than being a pure engineering decision, so we may find that zone operators require us
to use a blend of approaches e.g. differing by device or region, or expose it to being decided by a zone operator themselves. It
is likely, that over time, all of the above options may well be implemented.
Regardless of the mode of integration there needs to be a clear approach for the provision of SGX infrastructure - the hosting of SGX enclaves, interaction with enclaves and storage of encrypted data.
describes our thinking on SGX infrastructure design. The design includes additional links to documents that cover some of the underlying concepts, design decisions and a potential roadmap for delivery that we are currently considering.
assumes basic knowledge of SGX concepts, and some knowledge of Kubernetes for parts specific to that.
The SGX roadmap was reviewed by the TAC. The video of the session can be found here: