Skip to content

gjbae1212/fluent-bit-pubsub

Folders and files

NameName
Last commit message
Last commit date

Latest commit

5757e95 · Jan 18, 2020

History

14 Commits
Apr 16, 2019
Jan 18, 2020
Apr 21, 2019
Apr 19, 2019
Apr 14, 2019
Apr 14, 2019
May 29, 2019
Apr 14, 2019
Jan 18, 2020
Jan 18, 2020
Apr 15, 2019
Apr 21, 2019
Jan 18, 2020
Jan 18, 2020
Apr 13, 2019
Apr 13, 2019

Repository files navigation

fluent-bit output plugin for google pubsub

license Go Report Card

This plugin is used to publish data to queue in google pubsub.

You could easily use it.

Build

A bin directory already has been made binaries for mac, linux.

If you should directly make binaries for mac, linux

# local machine binary
$ bash make.sh build

# Your machine is mac, and if you should do to retry cross compiling for linux.
# A command in below is required a docker.  
$ bash make.sh build_linux

Usage

configuration options for fluent-bit.conf

Key Description Default
Project google cloud project id NONE(required)
Topic google pubsub topic name NONE(required)
JwtPath jwt file path for accessible google cloud project NONE(required)
Debug print debug log false(optional)
Timeout the maximum time that the client will attempt to publish a bundle of messages. (millsecond) 60000 (optional)
DelayThreshold publish a non-empty batch after this delay has passed. (millsecond) 1
ByteThreshold publish a batch when its size in bytes reaches this value. 1000000
CountThreshold publish a batch when it has been reached count of messages. 100

Example fluent-bit.conf

[Output]
    Name pubsub
    Match *
    Project your-project(custom)
    Topic your-topic-name(custom)
    Jwtpath your-jwtpath(custom)    

Example exec

$ fluent-bit -c [your config file] -e pubsub.so