Topics

Roadmap doc for project MAXIMUS

Mike Hearn
 

Hello,

 

For those of you interested in scaling to large numbers of hosted identities, Matthew has published a document for review on the various stages we are planning to get there in Corda Enterprise:

 

https://github.com/corda/corda/blob/9390b61a47164dd58696916bd8bf6476216d0cb3/docs/source/design/maximus/design.md

 

Please put comments here:

 

https://github.com/corda/corda/pull/4055

 

Executive summary:

 

  • There is an expressed desire to be able to on-board large numbers of small / low commitment users as first class Corda citizens. This is partly so they can do more with their Corda identity in future if they want to, even if in the beginning they just use an app as a service.

  • Currently the Corda node only supports one identity at a time. MAXIMUS is a project to support many identities per node, with some measure of horizontal scalability such that the node can scale up to handle higher traffic levels too.

  • Several different stages are outlined, with the first being delivered in Corda [Enterprise] 4. This first phase enables splitting Apache Artemis out of the node to run as a standalone MQ broker, with wire protocol extensions to support TLS SNI.

  • In the end state the MQ broker becomes the nervous system of a multi-service architecture (or ‘microservice architecture’ if you like). App workers switch identities on the fly as work comes in. They can be run on separate machines and thus this architectural direction opens up the possibility for physical isolation of hosted apps on e.g. dedicated hardware, behind different internal firewalls, different root accounts etc. The node becomes a logical rather than physical entity, made up of different cooperating servers.

 

MAXIMUS has a clearly defined scope. It does not include things like human interaction or allowing keys to be hosted outside of the super-node. That would be a separate project. MAXIMUS unlocks various other features like advanced flow control and being able to isolate apps from each other even if you aren’t hosting multiple nodes. However, those are also not part of this project.

This design has been discussed before, like at CordaCon Japan, so it won’t be news to some of you. There will also be a TAC review session on MAXIMUS soon. For everyone else, comments to the PR please!

 

thanks,

-mike

Mehdi Bechiri
 

Hi Mike,

First of all, I'm super excited about Maximus ! Not necesseraly by its immediate purpose, but because it will enable some future features that I really need (mostly the "multiple node per entity" but it's not the topic here).

Anyway, quick question on the v4.0 architecture model to handle its deployment properly. For Enttity A and Entity B vaults, would it be :

  • Separate SQL Server instances (hosted on different hardwares) ?
  • Different databases in the same instance ?
  • Different tables in the same database ?

PS : I know SQL Server is CE related, but it can apply to h2 db too.


Thanks,
Mehdi Bechiri.
  •  

Mike Hearn
 

You’d need separate table namespaces. Most DB engines can host multiple isolated databases on the same server. Corda doesn’t know or care what else the DB server is doing – it just needs a JDBC connection string. As long as resolving that JDBC URL gives a blank space to put things in, the node should be happy.

 

 

 

_._,_._,_

Mike Hearn
 

Matthew pointed out that we should clarify something.

Today the word "node" has a simple meaning - it's the thing that starts when you run Corda.

In a post-MAXIMUS world we will be redefining what the word "node" means slightly. The new definition will be something like, "the installation of Corda for your organisation". A node may be composed of multiple servers working together. This is already happening with the float/bridge/separate database server etc setup found in enterprise setups.

As part of this, apps will see what amounts to a virtualised node. The APIs can't change, so instead the current identity the node is acting as will change between runs of the app. Because of things like @CordaService this implies that apps may need slight adjustments to work efficiently on multi-identity nodes.

As we provide backwards compatibility, MAXIMUS nodes will be able to run existing apps. However not all the efficiency gains may be obtained in that case (it's a bit early to tell). If necessary the targetVersion mechanism will be used to let apps label themselves as having been tested against multi-identity nodes. Apps that target newer versions of the platform may need to provide a bit more info like whether a service is scoped to identity or process lifetime, things like that.

More info will be provided as this capability is developed.

Mark Simpson
 

@Mike

I’m wondering what your thinking is in terms of the concept of ‘serverless’ corda. The ability to deploy cordapps by just deploying jar(s) via an api into a hosted ‘cordas OS’  run under a specific identity is very appealing.
Having used amazon lamba to literally paste code into box on a webpage was very liberating.
Be great to hear what you think about taking it this far !
 

On 15 Oct 2018, at 11:40, Mike Hearn via Groups.Io <mike@...> wrote:

Matthew pointed out that we should clarify something.

Today the word "node" has a simple meaning - it's the thing that starts when you run Corda.

In a post-MAXIMUS world we will be redefining what the word "node" means slightly. The new definition will be something like, "the installation of Corda for your organisation". A node may be composed of multiple servers working together. This is already happening with the float/bridge/separate database server etc setup found in enterprise setups.

As part of this, apps will see what amounts to a virtualised node. The APIs can't change, so instead the current identity the node is acting as will change between runs of the app. Because of things like @CordaService this implies that apps may need slight adjustments to work efficiently on multi-identity nodes.

As we provide backwards compatibility, MAXIMUS nodes will be able to run existing apps. However not all the efficiency gains may be obtained in that case (it's a bit early to tell). If necessary the targetVersion mechanism will be used to let apps label themselves as having been tested against multi-identity nodes. Apps that target newer versions of the platform may need to provide a bit more info like whether a service is scoped to identity or process lifetime, things like that.

More info will be provided as this capability is developed.

Mike Hearn
 

Pretty sure AWS Lambda runs on servers 😉

 

Yes I’d love to get us to the point where developers inside an organization can trivially deploy apps to a node via a self service API. That’s going to be essential to reach the “3 firms, 3 devs, 3 days” goal I outlined in my CordaCon talk.

 

Some Corda users are very conservative. To get there we’ll need a few components:

 

  1. A way to deploy software to the node in a controlled manner. Our current bet: teach the node about Maven repositories and how to resolve from them. Then you can point your node at an internal Artifactory that is connected to your CI system, such that the release process is gated by the standard code review and CI gates.

    That way, a company can know that the apps the node has access to have all come through a formalized release process and they can of course mirror Maven Central to ensure all the dependencies are snapshotted, as many firms already do.

  2. RPCs and shell commands to tell the node “install app with coordinate <com.megacorp:megaapp>” and “upgrade app with coordinate….”, so the install/upgrade process can be permissioned to admins.

  3. Support for automatic upgrade of marked apps. For example, during iterative testing between firms, you don’t want to involve the node admins for every upgrade cycle, it should be automatic once a new release is pushed to CI. Once the app becomes more important to the business automatic upgrade can be disabled, and if it’s really important, it can be run on an isolated machine from other less critical apps or apps that are still in development.

 

So the goal is to have a smooth ramp from iterating-every-day to a more controlled production state where maybe more signoffs are required.

 

Ivan has a PR already that implements most of this, including running Maven resolution over flows. So you can host an “app store” on a BNO node and use the BNO Membership Service app to restrict access based on Corda identities, potentially in future it could integrate with tokens too – app store on the ledger? Sure, why not?

 

 

 

Mark Simpson
 

Thanks Mike, very helpful and I thought Amazon lambda ran on Jeff Bezo’s magical unicorns !
I would guess that Corda Serveless would run on a decentralised pool of servers that span clouds and corporate data centres.

On 15 Oct 2018, at 13:26, Mike Hearn via Groups.Io <mike@...> wrote:

Pretty sure AWS Lambda runs on servers 😉
 
Yes I’d love to get us to the point where developers inside an organization can trivially deploy apps to a node via a self service API. That’s going to be essential to reach the “3 firms, 3 devs, 3 days” goal I outlined in my CordaCon talk.
 
Some Corda users are very conservative. To get there we’ll need a few components:
 
  1. A way to deploy software to the node in a controlled manner. Our current bet: teach the node about Maven repositories and how to resolve from them. Then you can point your node at an internal Artifactory that is connected to your CI system, such that the release process is gated by the standard code review and CI gates.

    That way, a company can know that the apps the node has access to have all come through a formalized release process and they can of course mirror Maven Central to ensure all the dependencies are snapshotted, as many firms already do.

  2. RPCs and shell commands to tell the node “install app with coordinate <com.megacorp:megaapp>” and “upgrade app with coordinate….”, so the install/upgrade process can be permissioned to admins.

  3. Support for automatic upgrade of marked apps. For example, during iterative testing between firms, you don’t want to involve the node admins for every upgrade cycle, it should be automatic once a new release is pushed to CI. Once the app becomes more important to the business automatic upgrade can be disabled, and if it’s really important, it can be run on an isolated machine from other less critical apps or apps that are still in development.
 
So the goal is to have a smooth ramp from iterating-every-day to a more controlled production state where maybe more signoffs are required.
 
Ivan has a PR already that implements most of this, including running Maven resolution over flows. So you can host an “app store” on a BNO node and use the BNO Membership Service app to restrict access based on Corda identities, potentially in future it could integrate with tokens too – app store on the ledger? Sure, why not?