Tuesday, September 13, 2016

[Big Data] About Cloudera Utilization Report

Hi everyone,

After a long post, I'm back :)

In this post, I am gonna talk about Cloudera new feature : Cloudera Utilization Report.. How to enable it on test cluster...

We can already have a look about cluster situation from charts or YARN pages , now starting with CDH 5.7 , we can have a detailed report for it.

You can access that from CM main page -> Clusters -> Under General tab -> Utilization Report..

When you open the report, it show overall utilization of cluster in terms of CPU and Memory

And you can get into YARN and see the utilization for YARN applications, but in order to get that you have to enable "Container Usage Metric Collection" for YARN service, or you can see the error above.. But you can still see the Preemption Tuning page, which displays  graphs for each tenant that display the average steady fair share allocations against the average instantaneous fair share allocations and average overall allocations for CPU and memory allocations.

To enable container usage metric collection you can do following

Go to YARN service -> Configuration and do following

The user must get ticket from KDC server, in my test environment i set it to Oracle , note that ROOT is not working unless you defined it LDAP. But i dont recommend that.

After that Create CM container directory on hadoop to save container usage info. 
In order to that, do following.. On the YARN service -> Configuration -> Actions

After that restart YARN service. After that you can see no error for YARN metric collection

Please wait for a while for CDH collects values..

Ok, here is end of story. After a whle , i will make a post again to share my findings. I hope it will help you. 

Thanks for reading.

Enjoy & share.


1 comment :