This is part 2 of a series where I will go into a little more detail on how you use HeatMap Analyzer.

 

 

Filtering

Being able to filter and order objects can be useful, especially when there are a lot of them and you are trying to find where a problem may be originating.

Expanding the Filter option will display the following window.

 

 

This can be broken into four main sections. Each of these sections is effectively AND together when the filter is executed.

Filter by Attribute:  If this section is enabled it allows you to retrieve the top|bottom “n” objects ordered by avg|min|max attribute.

Object Type: For objects that expose a “Type” attribute (currently this is only LUNs and Disks). This option allows you to show only specific Object Types. For example if you only wanted to view Public RAID Group LUNs and Public Pool LUNs  then selecting these options will filter out all other types of LUNs. NOTE only object types are are present in the array will be available in the list, and the label that particular Object Types are given may vary depending on the version of Flare code.

Include: This option allows you to include objects based on whether the extended data for that object contain the text entered in this field, this may include Name, Owner, Type, Host (depending on what extended data is available for that object). This can be a comma separated list and is case insensitive. NOTE leading or trailing spaces will be included in the search.

Exclude: This option allows you to exclude objects based on whether the extended data for that object contains the text entered in this field, this may include Name, Owner, Type, Host (depending on what extended data is available for that object). This can be a comma separated list and is case insensitive. NOTE leading or trailing spaces will be included in the search.

 

So the following filter selections would display the Top 4 by average Queue Length Public Pool LUNs where the extended information includes SP A and excludes Hypervisor.

 

 

Once you have filtered the object list you are able to chart the objects against any of the attributes that are exposed by that object type.

 

In Part 3 we will look at the different chart types and what they may be used for.

Mat has agreed to do some posts on the basics of using the EMC HMA. In this episode, he’s looking at manually loading NAR files and other cool stuff. Enjoy.

 

This is part 1 of a series where I will go into a little more detail on how you use HeatMap Analyzer. In this article we go through the process of manually loading a NAR file, basic charting and HeatMaps.

At this point you should have the HeatMap Analyzer appliance configured and you may be wondering what you can begin to do with it. I wrote the original HeatMap tool so I could visualize what was occurring over an entire array or parts thereof at any one point in time. This evolution of that script is HMA. It  now allows you to look closer at individual components of the array and while this is something that can be done with the current EMC Analyzer tool set, it is limited by the amount of data that you can load at any one time.

 

Manually Load NAR File

Point your browser at the appliance http://1.1.1.1/HMA.html and go to the Configuration tab.

 

 

Expand the “Manual Load NAR Files” section and select the “Choose Files” button. Here you can select one or more NAR files from a single or multiple arrays.  Depending on the number of “objects” in the NAR file (disks, LUNs, Pools, Ports, etc), processing of each NAR file may take a reasonable amount of time. As a rough guide for every 100 objects in the NAR file it will take about 30 seconds to process (assuming the standard 300 data points for each object).  Some of my CX4-960 arrays that have 3000 odd objects take 10-15 minutes to process each file. NOTE: if you are processing large NAR files or lots of smaller NAR files then the browser may currently timeout waiting for the processing to finish. If this is the case you can monitor the process through the Server Status tab or from the command line of the appliance (see below).

 

Note: you should let all processing complete before you begin to draw HeatMaps or charts under the Analyzer tab, if you don’t you may get some unexpected results due to the non-granular locking that SQLite uses while updating the database. 

Monitoring the NAR file load process

If you want to monitor the processes that are working on your NAR file(s) there are some tools available on the Server Status tab.

For basic process monitoring expand the “Process Monitoring” section, and tick the “Auto Update” option. If you are currently processing a NAR file you will see the ProcessNAR.pl script running.

 

 

For more detailed monitoring you can view the logs created by each script. On the Server Status tab expand the “Logs Viewer” section, select the log file that you want to monitor and select “Auto Update” and “Auto Scroll”. You may want to combine this with the basic process monitoring to get a better picture of what is occurring.

 

Note: it’s worthwhile turning off Auto Update after you have completed using these tools as leaving it running will place unnecessary load on the appliance

HeatMaps

To draw HeatMaps for a particular array go to the HeatMaps tab.

 

 

Array: If you have multiple arrays you can select which array that you want to draw a HeatMap for.

Array Type: This gives a description of the array type (currently only CLARiiON and Data Domain types are supported).

Start/End: This is the Start and End date of the HeatMap. NOTE: Selecting a long duration for the HeatMap may take a while to process, and it is not recommended that you draw a HeatMap that spans more than several days without increasing the Average Interval.

Avg Interval: The number of seconds between each chart point.

Object List: This is a list of objects that you can select attributes from to chart and may include.

All LUNs / Storage Processors / Disks / RAID Group / Port / Thin Pool / CPU / Asynchronous Mirror / Snap Session.

The object that are shown will depend on what you have configured on your array (if you don’t have any Asynchronous Mirrors defined then you wont see this option).

Selecting one of these drop down lists will show a list of attributes that can be charted for this object.

 

 

Select an attribute and it will be added to the list of object/attributes that will be drawn in the HeatMap. Select which object / attribute combinations that you want to display in the HeatMap and then press the “Draw HeatMap” button. The HeatMap configuration pane can be hidden by clicking on the top arrow in the top left corner.

 

 

The way the HeatMap works is by defining a minimum, a maximum and a skew for each object and attribute. These can be modified in the Configuration Tab under the HeatMap Attribute Normalization.

Analyzing the Data

For more detailed analysis of individual objects in the array go to the Analyzer tab. On this tab you will initially see three panes. The first left hand pane allows you to configure and draw charts for various objects, the second upper right pane gives global details for the array and the third lower right pane gives you details for the selected object.

 

 

Array: If you have multiple arrays you can select which array that you want to draw a HeatMap for.

Array Type: This gives a description of the array type (currently only CLARiiON and Data Domain types are supported).

Object Type: The type of object to chart (ie Storage Processor, LUN, Disk, etc)

Start/End: This is the Start and End date of the chart. .

Filter: Filter options that allow you to include / exclude objects based on set criteria (I’ll go into more detail about this later).

Object: The object(s) that you want to chart.

Attribute: The metric that you want to chart for the object, each object type exposes a different set of attributes.

Chart dependent options:

Line Charts, Stacked Charts and Tables

Avg Interval: The number of seconds to average the data points out to, leaving as zero or blank will process the data as is.

Pie Charts

Max Slices: The Maximum number of slices to draw in a Pie chart. If there are more than “n-1” objects  then extra objects will be grouped under “Other”.

Distribution Charts

Minimum: The minimum number to chart.

Maximum: The maximum number to chart.

Intervals: The number of intervals to chart.

Chart Type:

Line: A standard line chart, multiple object can be selected and charted against one another.

Line by Day: A standard line chart where each day is stacked against each other, while you can select multiple object it’s not recommended as it makes the chart difficult to interpret.

Stacked: A stacked line chart, this option is available where the attribute being graphed is able to be aggregated (ie MB, KB or GB rather than % Utilization), again multiple objects can be selected.

Pie: A pie chart, this will draw a pie chart for all of the objects for the selected “Object Type”.

Distribution: A bar chart showing the % of values that fall between certain ranges.

Table: A text table of the data.

Connectivity: A Connectivity / SanKey chart. This show how each of the objects in the array are connected. It is only avaliable for CLARiiON devices.

Once you have selected the Object Type, Object(s) and Attribute that you want to chart, click the “Graph” button and chart will be drawn for that selection.

 

Depending on the type of chart drawn you may have various controls that allow you to explore the data more closely. Line style charts have the slider control at the bottom which allow you to focus on a subset of the data. Tables allow you to select all the data so it can be copied. Connectivity charts allow you to control the depth of the chart. And the attribute that controls the weight of each of the connections, all chart types except Tables allow you to save the chart as a JPEG.

 

In the next article I will go into more detail regarding Filtering and the different Chart Types