How to install Flume on Linux

Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data.
For more information see Flume Installation

First you need to update your Yum repository:

    cd /etc/yum.repos.d/
    sudo wget http://archive.cloudera.com/redhat/cdh/cloudera-cdh3.repo

In order to install Flume node:

    sudo yum install flume-node -y

In order to install Flume master:

    sudo yum install flume-master -y

Congratulations! You have installed Flume.
Before start using it, the first step you must do, is to tell each node about its master

    sudo su -l flume --shell=/bin/bash
    cd /etc/flume/conf.empty/
    cp flume-site.xml.template flume-site.xml

Edit flume-site.xml and set the value of flume.master.servers to be the name of your master server name (use the following command on the master server uname -n).
exit

Start your Flume node:

    sudo /etc/init.d/flume-node start

Start your Flume master:

    sudo /etc/init.d/flume-master start

To validate that that all nodes configured to the properly master, go to the master URL address at port 35871 and validate you see all your nodes listed there (e.g. http://webserver-name:35871/flumemaster.jsp).
You can check it by entering a simple configuration setup in the master web GUI:

some_node :  tail("/tmp/flumeInput.txt") | text("/tmp/flumeOutput.txt") ;

This will configure the node to tail an input file and write the lines into an output file.
Good luck!

Leave a Reply

*