Error DataReader.cc:224 unexpected mdsd ack status: ACK_DECODE_ERROR, tag '0'

[harshita01398]

I am using td-agent_3.8.1-0_amd64.deb and fluent-plugin-mdsd-0.1.9.pre.build.master.71-td.amd64.gem

td-agent.conf is the default one.

@type forward port 25585 bind 127.0.0.1

<match mdsdNet.**>
@type mdsd
@log_level info
djsonsocket /var/run/mdsd/default_djson.socket
acktimeoutms 5000 # milli-seconds
# Convert hash value to valid json. The => are replaced with :
convert_hash_to_json true
num_threads 1
buffer_chunk_limit 8m
buffer_type file
buffer_path /var/log/td-agent/state/out_mdsd*.buffer
buffer_queue_limit 128
flush_interval 1s
retry_limit 3
retry_wait 10s

However I see that the fluent events sent for logger.info get correctly pushed to mdsd however all other log types (debug, error, exception) are suppressed. In the td-agent.log, I saw below errors:

2021-05-25 12:07:36 +0000 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg=2
2021-05-25 12:07:36 +0000 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg=3
2021-05-25 12:07:36 +0000 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg=0
2021-05-25 12:07:36 +0000 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg=15
2021-05-25 12:07:36 +0000 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg=0
2021-05-25 12:07:36 +0000 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg=1
2021-05-25 12:07:36 +0000 [warn]: #0 incoming chunk is broken: host="127.0.0.1" msg=1
2021-05-25T12:09:39.3500460Z: error DataReader.cc:224 unexpected mdsd ack status: ACK_DECODE_ERROR, tag '0'
2021-05-25T12:12:45.7766050Z: error DataReader.cc:224 unexpected mdsd ack status: ACK_DECODE_ERROR, tag '0'
2021-05-25T12:12:49.5995980Z: error DataReader.cc:224 unexpected mdsd ack status: ACK_DECODE_ERROR, tag '0'

2021-05-25 12:07:21 +0000 [warn]: #0 emit transaction failed: error_class=TypeError error="no implicit conversion of Integer into String" location="/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.11.5/lib/fluent/match.rb:137:in match?'" tag=0 2021-05-25 12:07:26 +0000 [warn]: #0 emit transaction failed: error_class=TypeError error="no implicit conversion of Integer into String" location="/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.11.5/lib/fluent/match.rb:137:in match?'" tag=34

[essale]

Any update regarding this case ?
i am having the same issue as well

[hitenjava]

Any update? I'm facing the same issue.

[hitenjava]

Update - Adding it just in case if someone comes across similar issue. I was able to fix by using simple logging with color disabled. Colors was causing addition of extra characters.
Something like below:
builder.AddSimpleConsole(i => i.ColorBehavior = LoggerColorBehavior.Disabled).AddDebug();
instead of:
builder.AddConsole().AddDebug()

[andyatwork]

Disabling colorization is not always possible, especially when running third-party workloads. Opened this issue to ensure the plugin handles (encodes or sanitizes) such characters safely and successfully: #79

[WqMsft]

Update - Adding it just in case if someone comes across similar issue. I was able to fix by using simple logging with color disabled. Colors was causing addition of extra characters. Something like below: builder.AddSimpleConsole(i => i.ColorBehavior = LoggerColorBehavior.Disabled).AddDebug(); instead of: builder.AddConsole().AddDebug()

Added this code to program class where our apis are being monitored by Geneva. Still receives the Decode error.

[WqMsft]

Update - Adding it just in case if someone comes across similar issue. I was able to fix by using simple logging with color disabled. Colors was causing addition of extra characters. Something like below: builder.AddSimpleConsole(i => i.ColorBehavior = LoggerColorBehavior.Disabled).AddDebug(); instead of: builder.AddConsole().AddDebug()

Confirmed by the original user through teams that this fix is not a solution for this error. They don't have this code anymore but it got auto-resolved.