Anyone tempted to write a custom database should ask if they’re prepared to be in the database maintenance business.
Because that’s where they’re heading, for a really long time.
For anyone seeking to write their own database, I have one recommendation: the Tokyo Cabinet Library.
Tokyo Cabinet abstracts away all the needs of writing your own serializers and deserilizers for binary formats. You can have hashtable databases , B+ trees and everything else all prepared for you under one roof.
Of course that is if you have brain, and don’t use a text storage format like JSON. If you use shit like JSON and YAML — and add potentially hundreds of millisceonds of parsing time just to serialize data from text into machine-readable binary, then please submit your name and address to my council so we can get rid of you when we own the world.
Cleansing of the undesirables aside (seriously, give me ONE good thing about text storage formats! They are EXCHANGE formats, not STORAGE formats!), Tokyo Cabinet is written in C so you can easily bind it with SWIG. But there’s probably bindings around if you look.
Never heard of this. Thanks for mentioning it!
Comparing cost to AWS Aurora is unfair. Give us the self host price, and compare to that.
Also, they should have tried Scylla or Cassandra. It’s very scalable and handles a lot of writes.
Why does everyone on the planet hate sql nowadays?
If your not dealing with billions of transactions why not?
Maybe it reminds them of Oracle.
Aren’t time series databases like Prometheus pretty good at storing this kind of data?
I don’t believe Prometheus supports geospatial data. Two minutes of googling though, so I could be wrong.
They’re just storing doubles in their own format too. I’m not sure if they even need any spatial lookups on the data, they didn’t mention anything about that in the article. Maybe they do that in-memory?
I imagine a delta encoding scheme similar to what the time series DBs use would work well for the geo points. Maybe even a delta-of-delta encoding for things like ships which move very consistently.
It’s probably not worth it given how small they’ve already go their data. But it is fun.
If you’re using Java you can use eclipsestore. Seems like a good project