RFC-6370: foyer layer

[MrCroxx]

Which issue does this PR close?

Closes #.

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

[Xuanwo]

We ususally don't need an RFC for new layers, but I'm happy to see one!

[WenyXu]

@MrCroxx @Xuanwo Nice work! We’re also looking into a caching layer for OpenDAL in our project. Do we already have any plans for caching strategies in the range get? Maybe we can open a tracking issue and work on it together.

A hybrid approach sounds good to us—for example, caching whole objects if they’re small enough, and using page-aligned slices for bigger ones. It might also make sense to have some cache control options for writers(event readers?)—for example, being able to mark certain files so they don’t get cached at all.

[flaneur2020]

@WenyXu +1 for the page-aligned caching scheme. imho, it should be a must-have for serious production workloads.

i’m currently working on integrating Slatedb with OpenDAL in slatedb/slatedb#1104. the main blocker at the moment is to add a page-aligned caching support to OpenDAL.

currently, Slatedb has its own page-aligned caching middleware around object_store (in https://github.com/slatedb/slatedb/blob/main/slatedb/src/cached_object_store/object_store.rs) and kinda works well in production workloads. I’d like to port that implementation into the Foyer layer after #6366 is merged.