Data mining from cloud

There are few playbooks which help us to get information from inventory. I mean to get information from applications in inventory ;)

1. Grep logs

From time to time we need look for some “statistics” in log => for example how often caldav is used, or how many times users log into application.

For this purpose is designed playbook grep_in_logs.

ansible-playbook -i easy_cloud ./playbooks/cloud_data_mining/grep_in_logs.yml -e "questions=['your grep expression']"

It run zgrep command in each application directory in every server in inventory. Results are gather to file ./playbooks/cloud_data_mining/data/grep_in_logs.csv.

2. Run ruby script in applications

For complex statistics in-app is required to create ruby script which is executed in rails runner directly in application.

So firstly you need create this ruby script in ./playbooks/cloud_data_mining/templates/run_in_app/.

Use naming convention ####_name => #ID of task in esko#_"some human readable name".rb.j2 for better identification.

Ruby script has also strictly structure:

Ruby script

First line of line must be comment with CSV header = this first line is used for generation final file.

Your custom ruby code should be below line

# ======  HERE WILL BE YOUR CODE

please see example.rb.j2 !

Execution

ansible-playbook -i easy_cloud ./playbooks/cloud_data_mining/run_in_app_and_get_csv.yml -e "statistics=['your_script']"

“your_script” is name of your ruby script in ./playbooks/cloud_data_mining/templates/run_in_app/. !!! Its only name of file, without extension.

Output

For every statistic script (ruby script) is CSV file generated under ./playbooks/cloud_data_mining/data/ folder.