tl;dr know inputs, outputs, and tags, and you're gucci
If you've already seen Part 1, then you already know how to send logs over from rails or sinatra applications, with custom tags. Now all we have to do is match incoming logs with tags, and output them, with the fluentd server. Simple, right?
You'll need to install fluentd through the ruby gem or
through a RPM/DEB/.dmg package. I've done both, and the package is probably the easiest way. But let's go through installing on
ubuntu first.
If you're running Ubuntu, all you have to do is install the DEB package following these instructions. You're config file will be located at /etc/td-agent/td-agent.conf, and you can run
the service like this: $ sudo service td-agent start or like this: $ /etc/init.d/td-agent start
Installing with the ruby gem is trickier, but do-able (you'll
likely run into more random errors this way though, I've found). The instructions on the fluentd docs are a little vague, so I'd
like to elaborate on each of the steps.
After installing the gem, you'll run $ fluentd --setup ./fluent to
make the directory where your fluentd configuration will be. By default the config file will be at ./fluent/fluent.conf. Keep in
mind that wherever you run the setup command will be where the fluent directory is created (something that took me waaayy too long to figure out). To start the fluentd service, run
$ fluentd -c [config-file] So, for example, if your config file is at ~/fluent/fluent.conf, you'll run
$ fluentd -c ~/fluent/fluent.conf You can add in options like "-vv" or "-v" to be more verbose (with -vv being the most verbose, and very helpful for debugging)
**All configuration is done in your config file, which will be either fluent.conf or td-agent.conf depending on how you installed it. Make sure you're editing the right file!
Now that you have a fluent server running, it's time to accept incoming logs. The first configuration will be the "input", which
is a source directive in fluentd. We have to tell the fluent server which port it should accept connections from, like this:
<source>
type forward
port 24224
</source>
This tells the fluent server to accept TCP port connections from port 24224. You can have as many "inputs", or source directives, as you'd like. There are a variety of input plugins that you can
choose from, although some may require some extra installation. For example, you can have one of your inputs be a file, and you
can tell fluent to tail a certain file and collect all those logs.
Keep in mind that if you choose to tail a file as one of your source directives, you'll need to give td-agent read access to those
logs, since I've found that a lot of nginx and syslogs are set to 640, which td-agent will (rightfully, perhaps) complain about
Now that fluentd is accepting logs, we can use the tags we chose when we sent the logs to choose where each of the logs will go.
For example, if you used the fluent-logger-sinatra gem, and you defined your logger like this: logger = FluentLoggerSinatra::Logger.new('myapp', 'default_log', '127.0.0.1', 24224) then all logs sent from that logger would be tagged with "myapp.default_log". In the fluentd configuration, we want to match those tags. So in your config file:
<source>
type forward
port 24224
</source>
<match myapp.default_log>
type file
format out_file
output_tag false
path /Users/sliu/log/myapp/default_logs/default_log
time_slice_format %Y%m%d%H%M
time_format %Y-%m-%d %H:%M:%S
utc
</match>
This tells the fluent server that any logs with the tag "myapp.default_log" should be written to the file at the specified path
(the last part of the path - default_log - will be the first part of the name of the file, with the rest being the time stamp, so logs will ultimately look like "default_log.01022015.log").
I chose to not include the tags in the logs (output_tag false) but including them is helpful for debugging (and you can match ** to
catch all unmatched tags). I also specified a time_format to make the time look better.
Outputting to file is probably
the easiest way to sort logs, but there are other output plugins like outputting to s3, a mongodb, or elasticsearch.
Those are the basic configurations for using fluentd. In another
post, I'll discuss how to send logs from one fluentd server to a remote fluentd server, and ssl encrypt those logs for security.
(There is no way yet of sending encrypted logs right from your app to a remote fluentd server). I'm feeling lazy right now, but hopefully I get to it soon.
A few things when setting fluentd up - if you want to write to a file, like in the example above, you need to give td-agent write access to those folders. If you want td-agent to make parent directories to a file folder, you can also give it execute access to a parent directory, and it will do that. If you installed a td-agent package (rather than use gem installation), the logs won't be output in stdout like they will if you used the gem. To see the logs, go into /var/log/td-agent/td-agent.log (which you might need sudo access to see). If somethings still not right and you can't figure out why, catch all logs that come into your sources with