Oskar Dudycz

Pragmatycznie o programowaniu

Never Lose Data Again - Event Sourcing to the Rescue!

2022-11-06 oskar dudyczEvent Sourcing


Relational databases are not losing data. They’re robust, consistent and secure! Are they?

Too often, we mix technical and logical concepts. Business transactions are not the same as database transactions. The fact that we can safely assume that information that we put into the database will be stored durable doesn’t mean that it won’t be lost.

We don’t even need to delete any data to lose precious business information. Each override replacing the previous state with the new one may already erase the valuable information. For instance, seeing that the shopping cart is empty, how will you know that it’s just a new shopping cart or someone added and removed a product? Such “pessimistic” scenarios are sometimes more valuable insight for the business than just knowing that everything went right and someone bought something eventually. In this example, knowing that someone put the product into the shopping cart but decided not to buy it can give us information about some gap or even an opportunity to “close the deal”. Yet, we need to have that information!

During my studies, I spent the first three summer breaks working as a shop assistant in a clothes shop. The most important trick I learned was to persuade customers to try clothes. After that, the chance of buying grew a lot. The same rule applies to e-commerce. The rate of abandoned carts is between 60-95%. That’s a lot. Those customers who had the intention to buy it and were so close to doing it but at the last phase resigned. If we store details of our business process, we can optimise and improve it. What’s more, targeting and focusing on the people we know are interested in is much more effective than trying to reach everyone. Such precision is much more cost-effective, and thus our business can benefit from it.

Event Sourcing, contrary to the standard approach, keeps all the facts that happened in our system. They’re stored as business events. We can look back and make analyses and enhanced diagnostics.

Events are a great source and input for future workflows: analytics, reporting or even machine learning. For instance, we can create a recommendation engine by knowing what products were bought together. We can also do various reporting and forecasting, e.g. having the number of sales as of the last month, we can see how it could look on the new pricing model. Such knowledge is critical in budgeting and process optimisations.

Nowadays, storage is cheap, but the information is priceless. We should take benefit of it. Keeping the business facts is much closer to how the real world works. In the real world, there’s no such thing as deletion. If we throw something into the trash bin, it doesn’t magically disappear. It’s just moved from one place to another.

It’s also worth noting that Event Sourcing is pretty often conflated with Event Streaming. Both names sound similar and are about events, but understanding the difference is essential. Event Sourcing is a storage pattern; it’s focused on capturing business facts and durably storing them. Event stores are databases and guarantee atomic writes, reading your own writes, and optimistic concurrency. So everything you’d expect from databases. Event Streaming is about data in motion. Tooling is focused on moving data from one place to another. This is the place where queues shine. Some of them have durability features, but they don’t provide a guarantee to make your decision based on the latest states. Read more in my other article Event Streaming is not Event Sourcing!.

InfoQ in their report shows Event Sourcing is in the “Late Majority” adoption phase. That means that it’s an established, common technique. Indeed it’s getting much more traction. Yet, in my opinion, it’s still a niche. That’s a chance for you as it means that you can get a competitive advantage by using it. You can do better than other companies, move fast and innovate. Using an event-driven approach can also help you build autonomous services and teams. That can help in making your development process even more efficient. Don’t wait to jump on the bandwagon; some empty seats are still left!

If you don’t know how to start, don’t be shy to contact me. I’m here to help. Check my training page. A workshop is the most effective way to jump-start. Try also my Introduction to Event Sourcing - Self-Paced Kit. When not to use Event Sourcing? I got you covered; I explained that in my other article here.



p.s. Ukraine is still under brutal Russian invasion. A lot of Ukrainian people are hurt, without shelter and need help. You can help in various ways, for instance, directly helping refugees, spreading awareness, putting pressure on your local government or companies. You can also support Ukraine by donating e.g. to Red Cross, Ukraine humanitarian organisation. You may also consider joining Tech for Ukraine initiative.

👋 If you found this article helpful and want to get notification about the next one, subscribe to Architecture Weekly.

✉️ Join over 1800 subscribers, get the best resources to boost your skills, and stay updated with Software Architecture trends!

Event-Driven by Oskar Dudycz
Oskar Dudycz For over 15 years, I have been creating IT systems close to the business. I started my career when StackOverflow didn't exist yet. I am a programmer, technical leader, architect. I like to create well-thought-out systems, tools and frameworks that are used in production and make people's lives easier. I believe Event Sourcing, CQRS, and in general, Event-Driven Architectures are a good foundation by which this can be achieved.