“From an object-relational mapping point of view, the need for traditional mapping decreases, although is not eliminated.”
CodexMicroORM is a light-weight, portable (NetStandard 2) library that solves many of the same problems that Entity Framework, Hibernate, Dapper and other ORM tools do, with a focus on performance and extensibility. The newest release, 0.5, hits both areas with even more goodness! In fact, we’ve cut down both memory usage and execution time on a range of workloads, with the intention of leading into the next major release with the addition of a companion add-on that can replace your use of a SQL database - or augment it, leading to a HybridSQL option, not just a binary SQL or NoSQL choice.
In the 0.2 releases, we used ConcurrentDictionary quite a bit – and still do to
a degree to make most operations thread-safe, unlike many other ORM
products. However, that collection type can be “heavy” on memory as evidenced
by our memory profiles which showed improved allocation rates once we switched
to a reader/writer locking system over plain dictionaries. Through this and
other changes, we see CodexMicroORM (aka CEF) drawing close to Dapper, which as
we note on our GitHub home
takes a different form than what many other ORM tools try to do, leading to
more verbose code.
This new benchmark is a derivative of one of our previous test suites, with the addition of 50,000 additional reads. Such a read-heavy workload is common for many mainstream apps. We offer the actual bits for these tests in our WPF test harness, as shown here:
We’ve effectively invalidated our previously posted benchmarks with the 0.5 release - but in a very favorable way for CEF!
In 0.5, we’ve improved collection types such as ConcurrentIndexedList<T> by adding estimated capacity, read/write locks and non-nullable constraints. This is an example of an object you can leverage in your own work (Apache 2.0 license) even if you’re not interested in using the rest of CEF. One of the drivers for changes like this in 0.5 was through testing we’ve been doing related to a new component…
If you’ve got kids (or have a good memory), you might be familiar with Dr. Seuss’ “There’s a Wocket in my Pocket.” (One of it’s characters is undoubtedly the inspiration for Zillow’s name!) Here’s another inspiration: “I like the Zable on the table…” – well, we’ve code-named our new HybridSQL extension “ZableDB.” It offers several benefits:
- Eliminate an entire architectural layer: the SQL database as something that needs to stand apart from your app. From an object-relational mapping point of view, the need for traditional mapping decreases, although is not eliminated – we still ideally model relationships between objects using surrogate keys (something that CodexMicroORM can do for us, regardless of the shape of our source object model). CodexMicroORM and ZableDB are nicely integrated, with ZableDB being another database provider that can be enabled simply:
CEF.AddGlobalService(new DBService(new ZableDBProvider()));
FileSystemPerObject.DatabaseDirectory = "ZDBBenchmark";
. . .
- Gain the performance benefits of an in-memory object-oriented database. With memory being cheap and cloud solutions making it easy to “dial up” resources, we can make the object models your application would ordinarily use something that can be quietly persisted.
- Your primary query language becomes LINQ to Objects. We offer automatic index creation and other optimizations that improve query performance.
- You might feel that having your data in ZableDB locks it away from outside use, making it less useful – however we’re including the ability to access your live data using an optional REST API, with OData as a standard interchange format. This means you can use tools such as Linqpad to query your live application data, with a security model included (row and column-level). In an up-coming article we’ll show an example of using PowerBI to visualize live application data using OData.
- Much as CEF is built on providers and services, ZableDB is extensible as well: storage models can be added or changed. For example, our initial implementation includes file system persistence, but we could add persistence to Amazon, Azure, HDFS, etc. in a “pluggable” way.
- You can mix SQL and NoSQL semantics. To support this, in 0.5 your choice of database provider can now be set by object type. This offers an ability to mix some objects that need SQL semantics with those which can use NoSQL semantics in the optimized memory-store.
To get an idea about how significant of a difference ZableDB can make, we’ve taken the benchmark shown above and implemented it using some early alpha bits. We’re able to achieve a solid 58 microseconds per operation (read-heavy, yes, but still “mixed”):
Now it’s true: your workloads might look different than what’s being benchmarked. However, it’s obvious it’d be hard to beat direct in-memory, in-process data access.
It’s also true that this type of solution isn’t going to be applicable to some situations. These include:
- Full ACID transactions are required. That’s not to say ZableDB won’t support transactions at all – we’ll be shipping multiple modes of support incrementally, with the default being relatively standard NoSQL semantics.
- Memory is at a premium. If you can’t take advantage of 64-bit, for example, you’ll be more constrained.
- If you can’t afford “start-up time.” A trade-off for ZableDB’s performance is when your application starts (be it a web app, service, or anything else), persisted data is brought into memory, which takes time. You can prioritize what is loaded, and control how much needs to be loaded before you can continue – or rely on the mix of SQL and NoSQL objects.
- Ability to scale out: the nature of ZableDB as an in-memory store means the preferred approach is to embed it, versus connect to it like a traditional SQL database. However, features like OData access will make for interesting opportunities.
I’ve already published the CodexMicroORM 0.5 update on GitHub and NuGet. Stay tuned for updates on ZableDB – either by signing up for notifications of new articles at xskrape.com, or watch for a 0.9 release on NuGet. At very least, ZableDB will offer an architectural option that you can start working with quickly and easily. Aside from ZableDB, CodexMicroORM continues to evolve as an open-source leader in performance and extensibility among ORM products.
Did you like this article? Please rate it!