Wednesday, October 21, 2015

Dashboard Design in Cloudera Manager and Creating Custom Charts with tsquery Language


In this post, i will talk about charts and dashboard design in Cloudera Manager  and i will show how to create a dashboard and how to add existing charts to that dashboards. After that we will create custom charts . We will use tsquery language which helps us writing chart-script in SQL-like style.

First, open CM mainpage and navigate to "Charts" on toolbar menu. Here we can manage existing dashboards or create new dashboards or build custom charts.

Lets create a new dashboard first. Choose "New Dashboard" from menu and give it a name. After that you can see an empty dashboard.

After that we can add built-in charts to our new dashboard. Charts are basically graphics written with tsquery language which helps to show some metrics . We can think this like "Top Activity" from Oracle Enterprise Manager Performance Page.

Ok, next we will add charts. For this purpose , we can open mainpage of the service that we want to see metrics from CM mainpage or we can add charts from mainpage of CM.

Let me add some metrics from CM main page. Open CM mainpage

We can add any chart with selecting "Add to Dashboard" option

After that we can see chart on our dashboard.

Lets create a custom chart with tsquery language. But before meet with tsquery language.

Tsquery Language & Metrics 

The tsquery language is used to specify statements for retrieving time-series data from the Cloudera Manager time-series data store. Its syntax is very like to SQL syntax , so we are familiar with it. We can select metrics from CM and display it as a chart.

Once you find metrics for your investigation, you can load it with tsquery.

For example , if we want to show current HDFS size we can search for hdfs metrics in Cloudera Documentation page, and see "dfs_capacity_used" works for you.

Here it is important to use exact version for your cluster in Cloudera Documentation. Because some metrics are deprecated or contains no data.

After that you can use this metric with chart builder. For this purpose open chart builder from CM toolbar.

Here, you can use query part as the follows . Only write like SQL query.

"SELECT dfs_capacity_used where roleType=DATANODE" 

After that press "Build Chart" and you can see the chart you just created. You can visualize it with the menu below.

Lets add another chart, here we are looking for memory usage for yarn applications. Search for memory in yarn metrics from Cloudera documentation and you will find following

You can create chart with this query:

"SELECT allocated_memory_mb WHERE serviceName = yarn" 

and see following chart for YARN pool usage.

You can use aggregate functions with tsquery and it helps you with writing.
When you type query in tsquery , CM tries to autocompletes your query , and you can use menubar for "WHERE" condition to manipulate chart.

At the end you can get a dashboard like this and it seems qood.

Last thing, currently dashboards are not secured, i mean open for everyone can logon to CM. So you can create custom dashboards for different management roles.

With this work, you can gather all relevant charts to a dashboard and you can easily follow what is going on your cluster.

Thanks for reading.

Enjoy & share.


No comments :

Post a Comment