How to install Flume on Linux
Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data.
For more information see Flume Installation
First you need to update your Yum repository:
cd /etc/yum.repos.d/ sudo wget http://archive.cloudera.com/redhat/cdh/cloudera-cdh3.repo |
In order to install Flume node:
sudo yum install flume-node -y |
In order to install Flume master:
sudo yum install flume-master -y |
Congratulations! You have installed Flume.
Before start using it, the first step you must do, is to tell each node about its master
sudo su -l flume --shell=/bin/bash cd /etc/flume/conf.empty/ cp flume-site.xml.template flume-site.xml |
Edit flume-site.xml and set the value of flume.master.servers to be the name of your master server name (use the following command on the master server uname -n).
exit
Start your Flume node:
sudo /etc/init.d/flume-node start |
Start your Flume master:
sudo /etc/init.d/flume-master start |
To validate that that all nodes configured to the properly master, go to the master URL address at port 35871 and validate you see all your nodes listed there (e.g. http://webserver-name:35871/flumemaster.jsp).
You can check it by entering a simple configuration setup in the master web GUI:
some_node : tail("/tmp/flumeInput.txt") | text("/tmp/flumeOutput.txt") ;
This will configure the node to tail an input file and write the lines into an output file.
Good luck!