Topics

Database support

Matthew@...
 

This initial screen cast is cool.  But, the obvious next question is how do I work with a database?


First thought:  what if the database is just another Func that services can call.  A "private" Func, that is defined in swagger (I'm reaching), and can only be used within this bundle?


OK, who wants to rip it to shreds?


-Matthew


-Matthew

David "Funcmaster D" Pollak
 

For any feature, we have to think about how it's going to be used during development, testing, and production (staging is a variation on production.

I'm thinking database access should be done (at least in JVM apps) via JDBC because there's a large set of ORM tools and most Java devs and most companies have their choice. Nothing about Funcatron should mess with this.

So, we need to deliver a set of JDBC credentials to the Runner with the request.

On the developer desktop, I'm thinking we just have the Docker container that has the mini Funcatron in it also have access to developer-local JDBC credentials.

Same for testing.

For production, we need to loop into Vault or whatever secrets management system is available on the container orchestration substrate.

I think we'll also need to extend the Swagger definition to make one or more DB connection available. Probably some named connection.

Then we surface the named DB connection via the Context parameter.

So, the function will look like:

public Foo apply(Bar param, Context context) {
  JDBC conn = context.db("default");
  return conn.doSomeQuery(...);
}

WDYT?

--
Funcatron, Simply Serverless http://funcatron.org
Lift, the simply functional web framework http://liftweb.net


On Mon, Oct 31, 2016, at 04:53 PM, Matthew@... wrote:

This initial screen cast is cool.  But, the obvious next question is how do I work with a database?


First thought:  what if the database is just another Func that services can call.  A "private" Func, that is defined in swagger (I'm reaching), and can only be used within this bundle?


OK, who wants to rip it to shreds?


-Matthew


-Matthew







Matthew@...
 

That seems like a good approach to me, but I'm not close enough to the new container substrates to really understand if we are missing any clever alternatives that they enable.  


Let me show this off to a couple people that know them well and get their feedback.

rajeev.cyrus@...
 

From a developers perspective nothing should dictate how the database is accessed. Since databases are stateful and functions are not the chosen database should be its own long lived thing. Completely outside of the funcatron world. IE; a database service, a database container, an external thing, flat files. Whatever. 

In my opinion all of this should be left in the hands of the developer, at most the devs would set their data access parameters via ENV variables that the functions read from to establish connections. 

The only major draw back I can see, from this approach or the JDBC approach is that we'd be creating new connections every time the function is activated and dropping them when it goes out of scope. Database connections are time expensive. 

What would be neat is someway to maintain a connection pool for a function or a group of functions. 

David "Funcmaster D" Pollak
 

Answers inline...

--
Funcatron, Simply Serverless http://funcatron.org
Lift, the simply functional web framework http://liftweb.net


On Fri, Nov 4, 2016, at 08:05 AM, rajeev.cyrus@... wrote:

From a developers perspective nothing should dictate how the database is accessed.


There's a wide gap between "dictate" and "offer tools for." Nobody is suggesting that there be one and only one way to do database access. What we are discussing is how Funcatron can *offer* functions a pre-built DB connection.


Since databases are stateful and functions are not the chosen database should be its own long lived thing. Completely outside of the funcatron world. IE; a database service, a database container, an external thing, flat files. Whatever. 

In my opinion all of this should be left in the hands of the developer, at most the devs would set their data access parameters via ENV variables that the functions read from to establish connections. 


Okay... so where are you suggesting the ENV vars be defined? Is there just one set of ENV vars or should it depend on the execution context (dev-time, test-time, production-time)?

The only major draw back I can see, from this approach or the JDBC approach is that we'd be creating new connections every time the function is activated and dropping them when it goes out of scope. Database connections are time expensive. 

What would be neat is someway to maintain a connection pool for a function or a group of functions. 


Which is pretty much what I proposed. The JDBC connection in my example would be part of a pool. It would be committed/closed on successful function return. It would be rolled back and closed on an exception.