Date   

Re: Changes related to networkmap interface for Corda 4.0

Remo Meier
 

The Cordite network map works fine for us both with Corda 4.0 and the RC branch of Corda 4.1.

Regards Remo


Changes related to networkmap interface for Corda 4.0

Mahesh Govind
 

Dear Experts ,
Is there an architectural or interface change for networkmap in Corda 4.0 .
Will cordite work for corda 4.0 
regards
mahesh


--
Co-Founder - Digiledge
+91 9591810218


Re: Corda, scripting languages and Graal

Mike Hearn
 

Yep, I tried it out last year and gave a demo of a prototype called "corda.js" at last year's CordaCon.

At the time to get callbacks from Java back onto the NodeJS thread required some deep hacking in the GraalJS internals, so I never released it. Since then the NodeJS interop has improved and now it's quite easy to do.

In fact, I've recently written a small open source tool that lets Java/Kotlin programs easily access NodeJS modules:

https://github.com/mikehearn/nodejs-interop

That tool puts Java/JVM sitting "on top" i.e. Java code gets control first. If you want to go the other way, with NodeJS sitting 'on top' and JavaScript being the first code that gets control, you can do that quite easily also out of the box. Read this article to learn how to handle threading in that case:

https://medium.com/graalvm/multi-threaded-java-javascript-language-interoperability-in-graalvm-2f19c1f9c37b

At the moment Corda RPC doesn't know anything about Graal/NodeJS out of the box, but it would be easy to add to the framework so RPC observations are switched onto the NodeJS thread. Then improving Corda/NodeJS interop boils down to polishing the edges, e.g. converting RxJava observables into RxJS observables or equivalents, making it easier to start flows by string name rather than expecting a Java Class type. Very small things. If there's someone who wants to go this route I can show you what to do to finish off the last 5% of the interop work.


Re: Corda, scripting languages and Graal

krishnathp@...
 

Mike, 
Thanks for this post. Has anyone tried Corda on GraalVM or Corda has any plan to support GraalVM?


Re: Cordapp Migrations (again!).

Farzad
 

I really should re-read my emails more before hitting send. Second sentence should start with:

"It may be beholden on the Cordapp developer to minimise coupling for systems that connect to that Cordapp,"


Re: Cordapp Migrations (again!).

Farzad
 

Thanks Mike. 

Agree with your closing statement. It may be beholden on the Cordapp developer to minimise coupling for systems that connect that user a Cordapp, such that when there is fundamental change in the States and Flows, the client is protected by having a decoupled and versioned interface that supported for backwards compatibility. In Cordite, our separation layer is Braid. Although states and flows can be exposed via that interface, we prefer providing wrapper services (in the general sense of the word) that hide the underlying implementation. Each interface provided on Braid is named (confusingly as a "service") which can be versioned by name e.g. "LedgerAPI_v1". I'd prefer the versioning to be more formally defined. 

Now, if there was a way for a Cordapp provider to (a) signal a new version of the Cordapp (b) have the means (perhaps within Corda itself) to distribute the cordapp (we've discussed on this group Maven repo providers), ideally via signalling mechanism that negotiates a suitable upgrade window for all relevant parties ... maybe we can have (in some cases only) a way of providing the Cordapp as part of a comprehensive managed service. 

Is this what you meant? Do we have a place to discuss the requirements for something like this? Perhaps here I guess.

In the meantime ...

Thanks for noting calculated properties (btw for anyone searching for it in the Corda docs use "calculated values"). They are very useful!

I wrote some tests and in summary, they mostly work: a v2 Cordapp can create a state which can be later consumed in a transaction with a v1 Cordapp on another node. 

It may be a useful exercise to enumerate the types of evolution in Cordapp state types. Here's a table for starters (I really hope text tables work well in groups.io!)

|---|--------------------|-----------------------------|----------------------------------| 
|   | A                  | B                           | C                                | 
|---|--------------------|-----------------------------|----------------------------------| 
|   | operation          | v1 state in tx with v2 node | v2 state in tx with v1 node      | 
|---|--------------------|-----------------------------|----------------------------------| 
| 1 | Add field          | Yes, if nullable            | Yes. Will need to run more tests | 
|---|--------------------|-----------------------------|----------------------------------| 
| 2 | Remove field       | Yes, should work            | Yes, with calculated properties  | 
|---|--------------------|-----------------------------|----------------------------------| 
| 3 | Rename field       | No                          | Yes, with calculated properties  | 
|---|--------------------|-----------------------------|----------------------------------| 
| 4 | Restructure fields | No                          | No                               | 
|---|--------------------|-----------------------------|----------------------------------| 
| 5 | Retype field       | No                          | No                               | 
|---|--------------------|-----------------------------|----------------------------------| 


unknownProperties, together with some additional processing in the Cordapp, will enable B3, B4. 
It doesn't really work row B5 and C5 (field retype).
I wonder if it would be useful, somehow, to present unknownProperties to the Cordapp, segregated by Cordapp version. That seems to be the missing bit for me.



On 13 May 2019, at 14:01, Mike Hearn via Groups.Io <mike@...> wrote:

Shall we continue this discussion on corda-dev, Fuzz?

Where we left off: there is a new feature added to Corda 4 called "calculated properties" that lets you add fields to your schemas that don't have c'tor parameters, i.e. are calculated by class methods rather than provided externally. The output is stored anyway in the serialised message, so it's accessible via the synthetic objects spun up by the carpenter. It has some relevance here.

Revisiting the generic string->map idea, one feature that might be very useful to add is support for a magic field called "unknownProperties: Map<String, Object>". If this field exists in your class, then the map is populated with any properties found in the serialised message that weren't in the local class. In a Kotlin data object, this field would survive being passed through "copy", in Java you'd be expected to copy it by hand if you write your classes to be immutable. The map values would be synthetic objects accessible via reflection. In this way new fields can be added in ways that can survive passing through old nodes (in some cases only) and thus (again in some cases only) a feature could start being used before everyone has upgraded. But only if it's OK for old app installs to ignore the new data. Probably, in many cases, it's not OK and they're expected to do things with it.

The bigger issue we're dancing around here is the desire to combine the agility of fully managed service offerings, with the decentralisation and control of blockchain apps. That's a much bigger problem than just serialisation.


Re: CE 4.0 - additionalP2PAddresses

Bogdan Paunescu
 

I believe there is a misunderstanding somewhere. If you're going for a hot-cold deployment as presented in the CE docs, only one node instance will be active at any given time, meaning a peer node will only be able to use one of the advertised p2p addresses to open a connection to it. In your case, if done properly, you just need to ensure that the primary node is started first for it to acquire the lock on the shared database, and the back-ups in the BCP can be set to try and start after, only becoming active if the primary fails and the lock on the database expires (see mutual exclusion configuration properties).

Bogdan


Re: CE 4.0 - additionalP2PAddresses

Sean
 

Hi Bogdan,
  The reason for asking for prioritization is that we want to have the back-up in the primary site scanned prior to the other back-ups in BCP, for example.
  Thanks.

\Sean


Re: CE 4.0 - additionalP2PAddresses

Bogdan Paunescu
 

Hey Sean,

No sure why you're concerned about prioritizing those addresses, but you shouldn't worry about that. Only one will(should) be usable anyway as having 2 or more nodes with the same identity active at the same time can cause all sorts of problems.

Bogdan


Re: CE 4.0 - additionalP2PAddresses

Sean
 

Got it. Thanks, Bogdan.
By the way, is it a way to prioritize the additionalP2PAddresses? That is, to round robin the addresses in a given order?

\Sean


Re: corda-finance-workflows packaged with jackson?

Shams Asari
 

Hey Qian

 

The finance-workflows jar uses Jackson for legacy reasons. It was a lot worse than this in Corda 3 which we tried to fix up for Corda 4, but fell short of removing it completely. However, we should be able to do this for Corda 5.

 

What do you mean by the “real” Jackson jar? Does it conflict with the version of Jackson that you wish to use?

 

Shams

 

From: <corda-dev@groups.io> on behalf of "qian yan via Groups.Io" <qianyan.lambda@...>
Reply-To: "corda-dev@groups.io" <corda-dev@groups.io>
Date: Monday, 20 May 2019 at 03:11
To: "corda-dev@groups.io" <corda-dev@groups.io>
Subject: [corda-dev] corda-finance-workflows packaged with jackson?

 


As the above picture, I see the corda-finance-workflows packages the com.fasterxml.jackson, which results in a dependency issues if I want to use the real jackson jar as my dependency. So why Corda inline the jackson? How to remove these class from my dependency graph?


Re: CE 4.0 - additionalP2PAddresses

Bogdan Paunescu
 

One thing that slipped my mind, but additionalP2PAddresses requires min platform version to be 4 in the network. 3.0 and earlier versions can't use it.

Bogdan 


corda-finance-workflows packaged with jackson?

qian yan
 


As the above picture, I see the corda-finance-workflows packages the com.fasterxml.jackson, which results in a dependency issues if I want to use the real jackson jar as my dependency. So why Corda inline the jackson? How to remove these class from my dependency graph?


Re: CE 4.0 - additionalP2PAddresses

Bogdan Paunescu
 

Sean,

1) Unfortunately I'm not familiar with the BCP scenario, but in what you describe, yes, it will be a race between back-ups to become active. I don't recall ever testing or seeing a hot-cold deployment with 2 or more back-ups.
3) Reading 1) it seems you already know about the mutual exclusion mechanism used by the nodes to determine whether they are active or not. This mechanism ensures only one is active, meaning only one can accept and/or initiate connections.  Therefore, the node initiating the connection will round-robin through the list of advertised p2p addresses and attempt connections until one is made. So, by the time a flow message has to be sent, and AMQP connection is already established.

Bogdan


Re: CE 4.0 - additionalP2PAddresses

Sean
 

Bogdan,
  Thank you for the quick reply. To further discuss the points raised earlier -
1. For the multiple back-ups, let's look at the BCP scenario. In the primary site, there is a Hot/Code pair, and the same for BCP, for a total of 4 instances for the same node. p2pAddress is set for one instance in the primary site, and additionalP2PAddresses are for the other three instances. Forget for a moment whether it is optimal configuration or not and just focus on how it would work. Reading thru the doc, it seems the Active updates its lease per updateInterval, and the back-ups compete for the lease per waitInterval. Is it a random race among the back-ups to become the new Active after the previous Active goes down (loses its lease)? 
2. Yes, we want to explore native Corda support first before going for a third-party load balancer.
3. I am a little confused. If another node wants to initiates a flow to our node, it has 4 addresses to choose from. How will it know which one is for the current Active? You said "whichever is used to open the connection" but there is no connection yet. Please help me here.
4. Depends on how 3 works.

\Sean


Re: CE 4.0 - additionalP2PAddresses

Bogdan Paunescu
 

Hi Sean,

The additionalP2PAddresses was added as an alternative to load balancers in a hot-cold deployment. So it should be used only in hot-cold configurations. Because in hot-cold there is only one active instance, there is no danger of messages being sent to multiple nodes. To answer your questions:
1) in hot-cold deployments you can have as many back-ups as you can afford, but 1 is sufficient.
2) additionalP2PAddresses is required only for hot-cold nodes and only when there is no load balancer.
3) Sending node wants to open a connection to a hot-cold one. Looks up its nodeInfo and tries to connect to it using the list of p2pAddresses (primary and additional). Messages are sent to only one address (whichever is used to open the connection)
4) In hot-cold, only one instance of the node can be active at a given time, there will be no duplicates from 3.

The only information on this configuration parameter is in https://docs.corda.r3.com/hot-cold-deployment.html and https://docs.corda.r3.com/releases/master/corda-configuration-file.html?highlight=additionalp2paddresses

Hope this helps.

Bogdan


CE 4.0 - additionalP2PAddresses

Sean
 

Hello,
  The note at the end of https://docs.corda.r3.com/hot-cold-deployment.html mentions a new node configuration parameter additionalP2PAddresses. I'd like to get some clarity on it.
1. The fact that the parameter name is in plural form seems to suggest that we can have more than one back-up instance.
2. I have not tested, but should the node-info now have more than one address, one for the primary p2pAddress and the others for additionaP2PAddresses?
3. How does the additionaP2PAddresses parameter work? When another node sends a message to our node at p2pAddress, will it also send to additionaP2PAddresses?
4. Since all instances of our node share the same network drive for Artemis files, will there be any duplicates resulting from 3 above?

We'd like to explore the additionalP2PAddreses parameter so any documentation on it would be appreciated.

Thanks

\Sean


Re: Cordapp Migrations (again!).

Mike Hearn
 

Shall we continue this discussion on corda-dev, Fuzz?

Where we left off: there is a new feature added to Corda 4 called "calculated properties" that lets you add fields to your schemas that don't have c'tor parameters, i.e. are calculated by class methods rather than provided externally. The output is stored anyway in the serialised message, so it's accessible via the synthetic objects spun up by the carpenter. It has some relevance here.

Revisiting the generic string->map idea, one feature that might be very useful to add is support for a magic field called "unknownProperties: Map<String, Object>". If this field exists in your class, then the map is populated with any properties found in the serialised message that weren't in the local class. In a Kotlin data object, this field would survive being passed through "copy", in Java you'd be expected to copy it by hand if you write your classes to be immutable. The map values would be synthetic objects accessible via reflection. In this way new fields can be added in ways that can survive passing through old nodes (in some cases only) and thus (again in some cases only) a feature could start being used before everyone has upgraded. But only if it's OK for old app installs to ignore the new data. Probably, in many cases, it's not OK and they're expected to do things with it.

The bigger issue we're dancing around here is the desire to combine the agility of fully managed service offerings, with the decentralisation and control of blockchain apps. That's a much bigger problem than just serialisation.


Re: Containerization in Corda

Ryan Gledhill
 

Brilliant article Mike, thank you.

I appreciate the insight too, it is relatively easy to fall into the trap of oversubscribing resources too early on.
I expect scalability will not become an issue for some time for 99% of CorDapps, but it's worth keeping in mind and designing for it as early on as is possible.

Thanks again.


Re: Containerization in Corda

Mike Hearn
 

Apologies for never getting around to answering this one, Ryan.

The basic idea for horizontal scalability is to let admins split flow execution out into separate JVMs that share the same backend database, then use the MQ broker to load balance flows between them. Things like RPC execution, scheduled states, the shell, services etc remain in the "main" JVM process, but that doesn't do much work anymore. Then you can add more flow workers to add capacity, as long as the work shards amongst flows nicely (as it probably does).

However it's not clear to me that most users actually need horizontal scalability. Modern servers are vast and cheap, to such an absurd extent that our intuitions about what's reasonable are often out of date. Back of the envelope calculations can be very surprising. I've written a bit on this topic in the second part of this essay:

https://blog.plan99.net/vertical-architecture-734495f129c4

See the section labelled "Vertical architecture". It has sample prices for modern dedicated hardware (non AWS/Azure instances). 

For most users I bet if they load test Corda Enterprise with their app, and look at how much traffic they'll realistically be handling in the next 5 years, one big machine is enough. There's no shame in that. Recall that a modern 1U server rack can easily be equivalent to a complete rack of machines just 15 years ago, e.g. 48 cores, a terabyte of RAM. It's a stupendous amount of power - hardware has grown much faster than most business problems have.

541 - 560 of 1475