Apparently deallocation of direct-allocated ByteBuffers is bit tricky, since underlying off-heap buffers are NOT necessarily freed in timely manner, partly since there is no mechanism to force deallocation. As such they might only be collected when GC kicks in.
However there is apparently a way to force freeing using sun.misc.Unsafe. If so, we should probably try to use that mechanism to reduce likelihood of OOMEs caused by delayed deallocation.