Posts Tagged ‘E2E’

How to setup log rotation on Flume




Flume can be used to rotate logs file into a regular file system. Examples: In BE mode (Best Effort) exec config remuscl2-vm.atomant.net ‘tail("/tmp/error.log")’ ‘agentBESink("remuscl1-vm.atomant.net",35888)’ exec config remuscl1-vm.atomant.net ‘collectorSource( 35888 )’ ‘collectorSink("/tmp/nlog/%Y/%m/%d/errorlog", "mylog_%{host}_%H_",60000 )’ in E2E (End to End) exec config remuscl2-vm.atomant.net ‘tail("/tmp/error.log")’ ‘agentE2ESink("remuscl1-vm.atomant.net",35888)’ exec config remuscl1-vm.atomant.net ‘collectorSource( 35888 )’ ‘collector(600000) {collectorSink("/tmp/nloge/%Y/%m/%d/errorlog", "mylog_%{host}_%H_",600000 )}’ Explanations: collectorSink […]






How to use the Flume E2E mode




E2E provides the best reliability mode: You can tune the reliability level of the agents, simply by specifying a different kind of agent sink. There are three levels available. One of them is the E2E (End To End) mode, which relies on an acknowledgement, and will retry if no acknowledgement is received. E2E limitations: E2E […]