Timestamp sub-second resolution is lost in the to_msgpack

[robbiezhang]

Sub-second resolution is supported after fluentd v0.14. However, the to_msgpack will only keep the unix second.

fluentd-plugin-mdsd/src/fluent-plugin-mdsd/lib/fluent/plugin/out_mdsd.rb

Line 72 in 9bacb00

[tag, time, record].to_msgpack

A workaround is to us to_msgpack_ext to serialize the time.

[tag, time.to_msgpack_ext, record].to_msgpack

And de-serialize it in the emit.

        def write(chunk)
            if use_source_timestamp
                chunk.msgpack_each {|(tag, time, record)|
                    # Ruby (version >= 1.9) hash preserves insertion order. So the following item is
                    # the last item when iterating the 'record' hash.
                    record[emit_timestamp_name] = Time.at(Fluent::EventTime.from_msgpack_ext(time).to_r)
                    handle_record(tag, record)
                }
            else
                chunk.msgpack_each {|(tag, record)|
                    record[emit_timestamp_name] = Time.now
                    handle_record(tag, record)
                }
            end
            @log.flush
        end

[fatal10110]

@robbiezhang This example will not work since there is no Fluent::EventTime module in min supported version of fluentd (0.12)

[kenans]

Wonder if there's any update about this thread? The MDSD Fluentd plugin still seems to have the sub second precision issue.