Re: CI times

Mike Hearn

I think we’re in agreement – you identify two issues:


  1. Our unit tests are too slow because our mocks are not mocky enough.
  2. We run more tests than necessary because we don’t trust TeamCity or Gradle to reliably identify what’s changed, especially on big branch switches.


Those are the same issues discussed above.


I don’t think (1) is caused by any particular design pattern or lack of a framework though. There’s a simpler explanation – making a fast in-memory mock implementation of a thing is a lot of work. Mock vs real end up with different feature sets, must be developed independently, sometimes new features must be implemented twice etc. If you don’t do it things still work because of how the code is designed, it’s just slower, so it’s easy to skip.


I did write a mock for the networking layer, but we haven’t done it for the vault or database layer. This is partly because we use JPA internally and partly because we expose it to application developers. We should examine what it’d take to mock the vault. There are a few questions:


  • What’s the right layer to mock at? Should we mock out Hibernate and leave the vault code as-is? Or make the vault itself replaceable?
  • If the latter, what do we do with apps and code that wants to do db queries?
  • Is there a way to make the startup time of the existing db code faster? Any low hanging fruit we’ve missed e.g. do we need a production-grade connection pool at all, in test code?
  • How have other projects tackled these issues?


Every large Java projects that uses a database will have encountered the same frustrations, so I bet there’s a lot of info out there on how to optimise the combination of Hibernate+H2.

Join to automatically receive all group messages.