Skip to content

wrong json format produced message with single quote #18

@zjxlinux

Description

@zjxlinux

Description

When i am using syslog-ng and syslogng-kafka to push system logs to kafka Server. I notice that the messages are not json format, messages use single quote (which should be double-quotes ) and can not be recognized and parsed by logstash

reproduce

I use syslog-ng -F to debug and the logs be produced like this

1 2018-05-30T13:57:14.721616+08:00 www.test.com syslogng_kafka 32288 - - Message produced: {'FACILITY': 'cron', 'PRIORITY': 'info', 'HOST': 'www.test.com', 'PROGRAM': 'CROND', 'DATE': '1527659821', 'MESSAGE': '(root) CMD (/usr/lib64/sa/sa1 1 1)'}

On logstash server i received the _jsonparsefailure error
image

syslog-ng conf

source s_sys {
    file ("/proc/kmsg" program_override("kernel: "));
    unix-stream ("/dev/log" max-connections(512));
    internal();
};
destination syslog_to_kafka {
    python(
        class("syslogng_kafka.kafkadriver.KafkaDestination")
            on-error("fallback-to-string")
            options(
                hosts("xxx:9092")
                topic("system-log")
                verbose("True")
                display_stats("True")
                )
    );
};
log {
    source(s_sys);
    destination(syslog_to_kafka);
};

logstash conf

input {
  kafka {
    bootstrap_servers => "xxxx:9092"
    topics => ["system-log"]
    codec => json
  }
}
......

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions