Deterministic JVM (DJVM)
Tommy Lillehagen <tommy.lillehagen@...>
Contract Verification on a Deterministic JVM (DJVM)
Hi all,
Why do we need a deterministic sandbox?
It is important that all nodes that process a transaction always agree on whether it is valid or not. Because transaction types in Corda are defined using JVM byte code, this means that the execution of that byte code must be fully deterministic. Out of the box, a standard JVM is not fully deterministic, thus we must make some modifications in order to satisfy our requirements.
So, what does it mean for a piece of code to be fully deterministic? Ultimately, it means that the code, when viewed as a function, is pure. In other words, given the same set of inputs, it will always produce the same set of outputs without inflicting any side-effects that might later affect the computation.
What’s the current state of affairs?
In our pursuit of enabling deterministic contract verification in Corda, we have worked on a deterministic sandbox (DJVM) which will provide a safe environment for such verification methods to run in. As part of that work, I have just pushed a pull request for the first iteration of our DJVM work to our Corda repository; see corda/corda#3386. This will form the foundation for the future deterministic sandbox used in Corda. Before we get there though, the code needs thorough review, some further improvements and modifications, etc.
As you will see from the PR, the code in the top-level DJVM module has not yet been integrated with the rest of the platform. It will eventually become a part of the node and enforce deterministic and secure execution of smart contract code, which is mobile and may propagate around the network without human intervention. However, for now it stands alone as an evaluation version.
Preview and request for comments
We want to give developers the ability to start trying it out and get used to developing deterministic code under the set of constraints that we envision will be placed on contract code in the future. For details around the concept, implementation, constraints, scope, etc., please refer to the documentation site. A snapshot of the added documentation can be found on the PR: docs/source/key-concepts-djvm.rst.
NOTE: This is quite a meaty piece of work, and the PR is indeed quite large. I expect there to be quite a few comments and rounds of feedback, and for the PR to be up for a while before merging… and this is indeed the reason for my email – it would be great to get this in front of as many of our users as possible (i.e., you!), for us to perform a thorough review of the work covered by this PR, but also to start thinking about the follow-up work beyond this initial version of the sandbox. This is why I’m calling on anyone interested to take a look and provide feedback.
It should be mentioned that this work ties in closely with our development efforts on the SGX team, working towards a deterministic sandbox for running contract verification inside an SGX enclave. If you’re curious to learn more about that work, have a look at the following material:
Have a great weekend! Look forward to hearing your feedback on this.
Best regards, Tommy Lillehagen
|
|
Tommy Lillehagen <tommy.lillehagen@...>
Further to my previous email, it’s also worth noting the great work that Chris Rankin has been doing on making deterministic versions of our core and serialisation modules. Here’s a couple of links for reference:
Make sure you check those out as well if you’re interested in this topic.
|
|
Tommy Lillehagen <tommy.lillehagen@...>
I've posted two Medium stories on the topic:
|
|
richard@...
Thanks Tommy – this is great! There seems to be interest beyond the Corda community judging by the number of retweets and comments I received over the weekend when I publicised your first post. I’ll share it some more during working hours too. I think it’s really important that we get the eyes of the broader Java community on this ☺
Richard
Richard G Brown | R3. | Chief Technology Officer 2 London Wall Place | Floor 12 | London | EC2Y 5AU M: +44 7764 666821 | T: @gendal
From:
<corda-dev@groups.io> on behalf of "Tommy Lillehagen via Groups.Io" <tommy.lillehagen@...>
I've posted two Medium stories on the topic: · Deterministic Contract Verification
|
|
Chris Rankin
Writing a Verifiable Contract for a Deterministic JVM
So Corda will verify contracts within a Deterministic JVM - but this just begs the question: "How do you write a contract that
can run within the DJVM?"
We need to provide tools that will allow CorDapp developers to do this. And so to this end, we have been steadily chipping away at the Java APIs to identify a deterministic subset without APIs for accessing file systems, networks, the system clock, or any other
sources of external entropy. We have also created new modules
corda-core-deterministic and
corda-serialization-deterministic which contain just the deterministic functionality from
corda-core and
corda-serialization respectively. Contract classes which use only these deterministic Corda modules and Java APIs should then execute successfully on the DJVM. (Obviously those classes
not needed for contract verification can continue using the standard Corda modules as before.)
You can find more details about how we generate these deterministic modules
here, including instructions on how to integrate the deterministic Java APIs with IntelliJ.
All the code for these modules has been merged with the master branch in the Corda repository for everyone to examine, and we expecting to continue fine-tuning the modules' contents as we further develop the DJVM.
We will also be providing a template CorDapp project which will demonstrate how to build a CorDapp that has both deterministic and non-deterministic classes.
Please take this code out for a spin, and let us know what you think.
Cheers,
Chris Rankin
From: corda-dev@groups.io <corda-dev@groups.io> on behalf of Tommy Lillehagen via Groups.Io <tommy.lillehagen@...>
Sent: 15 June 2018 15:06 To: corda-dev@groups.io Subject: [corda-dev] Deterministic JVM (DJVM) Contract Verification on a Deterministic JVM (DJVM)
Hi all,
Why do we need a deterministic sandbox?
It is important that all nodes that process a transaction always agree on whether it is valid or not. Because transaction types in Corda are defined using JVM byte code, this means that the execution of that byte code must be fully deterministic. Out of the box, a standard JVM is not fully deterministic, thus we must make some modifications in order to satisfy our requirements.
So, what does it mean for a piece of code to be fully deterministic? Ultimately, it means that the code, when viewed as a function, is pure. In other words, given the same set of inputs, it will always produce the same set of outputs without inflicting any side-effects that might later affect the computation.
What’s the current state of affairs?
In our pursuit of enabling deterministic contract verification in Corda, we have worked on a deterministic sandbox (DJVM) which will provide a safe environment for such verification methods to run in. As part of that work, I have just pushed a pull request for the first iteration of our DJVM work to our Corda repository; see corda/corda#3386. This will form the foundation for the future deterministic sandbox used in Corda. Before we get there though, the code needs thorough review, some further improvements and modifications, etc.
As you will see from the PR, the code in the top-level DJVM module has not yet been integrated with the rest of the platform. It will eventually become a part of the node and enforce deterministic and secure execution of smart contract code, which is mobile and may propagate around the network without human intervention. However, for now it stands alone as an evaluation version.
Preview and request for comments
We want to give developers the ability to start trying it out and get used to developing deterministic code under the set of constraints that we envision will be placed on contract code in the future. For details around the concept, implementation, constraints, scope, etc., please refer to the documentation site. A snapshot of the added documentation can be found on the PR: docs/source/key-concepts-djvm.rst.
NOTE: This is quite a meaty piece of work, and the PR is indeed quite large. I expect there to be quite a few comments and rounds of feedback, and for the PR to be up for a while before merging… and this is indeed the reason for my email – it would be great to get this in front of as many of our users as possible (i.e., you!), for us to perform a thorough review of the work covered by this PR, but also to start thinking about the follow-up work beyond this initial version of the sandbox. This is why I’m calling on anyone interested to take a look and provide feedback.
It should be mentioned that this work ties in closely with our development efforts on the SGX team, working towards a deterministic sandbox for running contract verification inside an SGX enclave. If you’re curious to learn more about that work, have a look at the following material:
Have a great weekend! Look forward to hearing your feedback on this.
Best regards, Tommy Lillehagen
|
|
joel.dudley@...
Hi all,
Reviving this thread to let you know that the recent release of Corda 4.4 marks a major milestone for the DJVM:
If you get a chance, please download Corda 4.4, enable the DJVM, and give it a test run! We look forward to hearing how this affects your existing contracts - let us know on this thread or at productfeedback@.... Thanks, Joel
|
|
Sean
Hi Joel and team,
I ran Corda jdvm that came with CE4.4 on my CorDapp. First, the gradle script plugin passed with no complaints. And then I ran CordApp, i got the following error on an enum class. [ERROR] 15:27:02-0700 [flow-worker] corda.flow. - net.corda.node.internal.djvm.DeterministicVerificationException: NotSerializable: Deserializing obj as enum class sandbox.com.mycompany.membership.PartyRole with value CONTROLLER.0 but ordinality has changed, transaction: AE0324817AD08889C49FE5E270AC3086178ADB7D58E3ADC7AF3E50E5C41B195D {actor_id=user1, actor_owning_identity=CN=PartyC, O=PartyC, L=Dallas, C=US, actor_store_id=NODE_CONFIG, fiber-id=10000001, flow-id=50a2c0a5-e27e-4bf3-b070-d2c6816be7a6, invocation_id=2403e835-7f09-4eaf-a4a4-0773d036068b, invocation_timestamp=2020-04-16T22:26:51.472Z, origin=user1, session_id=e7d568c8-bf69-4d18-b995-ccc5545852dc, session_timestamp=2020-04-16T22:10:49.151Z, thread-id=736, trackingId=bc4b324d-ea3e-4ce7-adff-a8df3e3c9c94} Here is the simple enum class. @CordaSerializable
interface Role {
val roleName: String
val description: String
}
@CordaSerializable
enum class PartyRole(override val roleName: String, override val description: String) : Role {
CONTROLLER(roleName = Controller", description = "Party that is authorized to control the process."),
WORKER(roleName = "Worker", description = "Party that is authorized to do work only.");
override fun toString(): String = roleName
}
Not sure what makes DJVM unhappy. Is there a fix? Thanks. \Sean
|
|
joel.dudley@...
Hey Sean,
Thanks for reporting this issue. We looked into it and it turns out it was a bug on our end - SandboxEnumSerializer uses Enum.toString() to determine the instance's name, but PartyRole overrides the toString() method. Hence deserialization fails. Sorry about that. We've created a bug ticket to fix the issue. If you want to test it further, you should be able to work around it for now by removing your override of toString(). Thanks again, Joel
|
|
Sean
Ok, Joel. I removed my custom toString() and the flow moved past the previous error.
But now I have the following error. It looks like my existing data is not consistent with the default toString(). I may have to consume the existing states and start all over again. I will give that a try and continue testing from there. In the meantime, if you have any other ideas to solve the error, please advise. [ERROR] 10:02:53-0700 [flow-worker] corda.flow. - net.corda.core.node.services.VaultQueryException: An error occurred while attempting to query the vault: Failed to deserialise group OUTPUTS_GROUP at index 0 in transaction: net.corda.core.contracts.TransactionState (erased) -> data(net.corda.core.contracts.ContractState) -> Cannot construct evolution serializer for remote type
com.aurum.membership.PartyRole: Cannot resolve local enum member Issuer to a member of [CONTROLLER, WORKER] using rules {}
Full type information:
com.mycompany.membership.PartyRole(Controller|Worker) {actor_id=user1, actor_owning_identity=CN=PartyC, O=PartyC, L=Dallas, C=US, actor_store_id=NODE_CONFIG, fiber-id=10000001, flow-id=720adaac-dff8-4575-9d7d-16f7654ecae0, invocation_id=3d77e91f-81ec-4f23-9fbf-80ac307106f7, invocation_timestamp=2020-04-20T17:02:53.181Z, origin=user1, session_id=8c4ad9d2-e844-45b9-8bc7-423593377d01, session_timestamp=2020-04-20T17:00:51.095Z, thread-id=587, trackingId=e6537dd1-a987-4b5d-b749-0b51e3a1c92b}
|
|