Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I remember reading someone's comments about how instead of databases using their own data serialisation formats for persistence and then optimizing writes and read over that they should just utilize the FS directly and let all of the optimisations built by FS authors be taken advantage of.

I wish I could find that comment, because my explanation doesn't do it justice. Very interesting idea, someone's probably going to explain how it's already been tried in some old IBM database a long time ago and failed due to whatever reason.

I still think it should be tried with newer technologies though, sounds like a very interesting idea.



> instead of databases using their own data serialisation formats for persistence and then optimizing writes and read over that they should just utilize the FS directly and let all of the optimisations built by FS authors be taken advantage of.

The original article effectively argues the opposite: if your use case matches a database, then that will be way faster. Because the filesystem is both fully general, multi-process and multi-user, it's got to be pessimistic about its concurrency.

This is why e.g. games distribute their assets as massive blobs which are effectively filesystems - better, more predictable seek performance. Starting from the Doom WAD onwards.

For an example of databases that use the file system, both the mbox and maildir systems for email probably count?


ReiserFS was built on the premise that doing a filesystem right could get us to a point where the filesystem is the database for a lot of use cases.

It's now "somewhat possible" in that modern filesystem are overall mostly less broken about handling large number of small (or at least moderately small) files than they used to be.

But databases are still far more optimized for handling small pieces of data in the ways we want to handle data we put into databases, which typically also includes a need to index etc.


As far as I can remember MongoDB did not have any dedicated block caching mechanism in its earlier releases.

They basically mmap’ed the database file and argued that OS cache should do its job. Which makes sense but I guess it did not perform as well as any fune tuned caching mechanism.


Early MongoDB design choices are probably not great to call out for anything other than ignorance. mmap is a very naive view on how to easily work with data but it falls over pretty hard for any situation where ensuring your data doesn't get corrupted is important. MongoDB has come a long way, but its early technical decisions were not based on technical insight.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: