Hey Nigel, it’s awesome that you mention that, because it is actually not totally morphed.
I have been away from it for a while because I’ve been working all day and night (literally) for a few weeks now, with just enough sleep to get by. Complete madness. But also a lot of progress
So, what I think you refer to is that I wanted to look at implementing safe network storage under some abstractions, that are known in a C# microservice framwork called ServiceFabric, as
reliable queues and
reliable dictionaries. These are distributed structures meant to be replicated over nodes. But that would probably be a separate project from the EventStore, unless I find that I need the log structure for coordinating transactions between competing consumers in various processes. Then maybe, if it is performant enough, I might reuse the EventStore for convenience.
(ServiceFabric runs in Azure and is actually something Microsoft developed for their internal use. Azure SQL and such run on ServiceFabric. The system of my company runs on ServiceFabric.)
So when I do that, I could actually have my company’s entire system based on safe network
Would need to run the processes (the micro services) on servers of course, as we do today (no decentralized thing going on).
I wouldn’t actually contribute that much to what already exists by doing that (the concept of a queue or a dictionary is basically already there with a bit of help), but I would lower the bar for entry to a very large professional developer community if I could put it behind some familiar, dead simple to use and understand API:s - similar to what they are already used to be working with in the .NET stack.
Anyhow, that was just some additional information.
Fast access database, the topic. That would be key-value store for example, and that is essentially what it is in it’s current form.
As intrz says, a bit more information of the use case would help. Like for example what the access pattern is (complex queries or lookups?).
I’m not a db pro, I just work a lot with event sourcing, so I have gotten very familiar with that.
The idea of the DB app I’m writing, is that it will be running locally to continuously build projections out of the event streams. Projections are denormalised structures that are optimised for queries. So the combination of local access and denormalised structure, would render really fast access.
These projections could be saved back to the network, to make them available for global consumption. At a cost of additional storage needed for this reshaped data.
Event sourcing is not normally something you associate with fast access, rather with fast write - as it is append only of incremental changes.
But, with the addition of projections from the streams, you can shape the data into a form that is optimised for fast access.
So, well, yes I could definitely provide some information on if that technology, design and approach would be suitable for any given use case, and also on how to get started with it. When it comes to other forms of db:s I wouldn’t.