Utilities

HeatMap Analyzer

Current Version: 0.1 (download core and third-party components)

Release Notes / Documentation can be downloaded from here.

Purpose:

The heatmap script allows you to generate heatmaps of various metrics over time from NAR files generated from EMC CLARiiON / VNX arrays, as well as Data Domain performance data. You can store the info in a database too. And draw pictures and stuff.

 

EMC DIY Heatmaps

Current Version: 4.01c (download)

Release Notes can be downloaded from here.

Sample output can be downloaded from here.

Purpose:

The heatmap script allows you to generate heatmaps of various metrics over time from NAR files generated from EMC CLARiiON / VNX arrays.

 

EMC FlipFASTTiering

Current Version: 0.01 (download)

Release Notes can be downloaded from here.

Purpose:

Change CX4 / VNX FAST Tiering schedule from manual to scheduled and vice versa. Also provides basic reporting.

 

EMC ReplicationCapacity

Current Version: 0.01 (download)

Purpose:

A simple PERL script to provide information on CX4 / VNX replication storage use.

 

EMC CheckPoolLUNOwner

Current Version: 0.01 (download)

Purpose:

A simple PERL script to provide information on CX4 / VNX Pool LUN Ownership.

32 Comments

  1. Pingback: penguinpunk.net » EMC – DIY Heatmaps – Updated Version

  2. Pingback: penguinpunk.net » EMC – Listing pool disks with naviseccli

  3. I am trying out your heatmap for Clariion. This is a great tool but it does not work on my CX4 arrays. The error I get is “not a valid archive file” when I run the perl script. Any help you could provide would be greatly appreciated.

  4. Hey Johnny,

    would it be possible for you to post the command line that you are using, and the output that the script is producing?

    can you confirm the version of naviseccli you are using, the flare code the array is running, and the that analyzer enabler is installed on the array (if its trying to download NAZ files, rather than NAR files then this is probably the problem)

    If possible would you be able to mail one of the NAR files to me at matharvest@gmail.com

    thanks

    Mat.

  5. I updated my Navisphere CLI and that fixed me up! Thanks for the tip. I am loving this heatmap so far!

  7. I am getting a naviseccli error 256 for all my CX arrays when I attempt to use –get_drive_type. I have not been able to find any info on the actual error code. Any thoughts?

    Retrieving NAR file APM00071600488_SPA_2012-11-27_15-59-43-GMT_M06-00.nar Dated 11/27/2012 09:59:46 saving file to ./
    Downloaded NAR file successfully
    Querying array name
    Querying SP name
    Querying array for LUN characteristics
    naviseccli failed with error code: 256

  8. Found the issue. I am attempting to pull from a CX3-40 and -isthinlun -isspoollun -fastcache do not exist on the CX3-40. I edited the script until I can run it against my VNX’s.

  9. Hi,
    loving the heatmap utility (and so’s our storage guy :) )

    I was wondering if there was an easy way to get the data from a mysql db instead of the nar file?
    I ask because we currently grab the narfiles on an 2 hourly biases. The nar files “decoded” data is then uploaded in to a db.
    Thanks
    Dan

  10. Hi Dan,
    Mat and I have been talking about doing something like this for a while. I’ll let you know when we have a solution that you can try. Glad to hear the script is useful.
    Cheers,
    Dan

  11. Hi. I’m getting the follow error when try to dump the NAR file. Any ideas? Great script BTW..

    Fri Feb 8 16:48:48 2013 executing: “C:\Program Files (x86)\EMC\Navisphere CLI\naviseccli.EXE” analy
    zer -archivedump -data C:\Users\jhafner\NARFiles/APM00113800197_SPA_2013-02-04_03-37-26-GMT_M06-00.n
    ar -overwrite y -delim cm -enquote qu -object d -format on,pt,tt,u,tt,tb,tt,tb -timezone local -join
    -out tmp_analyzer_heatmap_d.csv
    Not a valid archive file.
    Fri Feb 8 16:48:48 2013 naviseccli failed with error code: 65280

  12. Hey Jason,

    its been a while since I have done any coding on the heatmap script (I need to stop upgrading our backup environment and get back to the important things) and I’m not near my work computer at the moment, so I cant look at the code.

    I can’t remember if there are issues with forward slashes in the path for the NAR file, if you can try moving the NAR file to the same directory that you are running the script from, and run the script again, just referencing the file name and no path and see how it goes …

    if that doesn’t work could you send me an e-mail with the command line that you are using, and possible the NAR file (matharvest@gmail.com)

    thanks

    Mat.

  13. I am very excited to be trying out this script, when I run it though I get the following error:

    C:\Temp>heatmap.3.021.pl –nar_file c:\temp\spa.nar –attrib d_utilization
    Adding file: c:\temp\spa.nar
    Processing NAR File:c:\temp\spa.nar
    Not a valid archive file (10).
    naviseccli failed with error code: 65280

    any help would be appreciated

  14. Small bit of feedback. The help output lists the s_date and e_date in mm/dd/yyyy format, while the code requires it in dd/mm/yyyy format. Otherwise thank you for an insanely useful tool.

  15. Thanks for the feedback Ronald – glad to hear the tool’s been useful. I’ll talk to Mat about the date requirement.

  16. Love the database option. Unfortunately does not work for me. Nar files are pulled, sqlite db created. I get this error when run against the DB:

    $ perl heatmap.pl –attrib s_iops –input_db heatmap.sqlite –s_date 30/07/2013 –e_date 04/08/2013
    Source Database: heatmap.sqlite
    Processing SP Attribute SP Total Throughput (IO/s)
    — WARNING no data —
    Generating heatmap file: heatmap.html

    Sqlite DB looks to have valid data when viewed with a sqlite browser.

  17. Hey Jeff,

    just wondering if it works if you don’t specify a start and end date?

    also could you get some dates out of the database (via sqlite browser) – this should be the poll time field, and check that they are correct – they should be in epoch format – so can use one of the online epoch converters, something like http://www.epochconverter.com/ and see if the dates are correct.

    happy for you to contact me at matharvest@gmail.com, if you could send me the database file, or I could send you some debug code to try and work out what is going on

    thanks for using the script, hope that we can work out what is going wrong

    thanks

    Mat.

  18. Love the tool. However, after loading nar data in to database, any call for lun attributes results in a divide by zero error.

    Y:\Array_Data\9325_Npv>perl ..\heatmap.4.0.1b.pl –input_db saptest.db –attrib l_iops
    Source Database: saptest.db
    Processing LUN Attribute LUN Total Throughput (IO/s)
    DBD::SQLite::db selectall_arrayref failed: no such table: data_l at ..\heatmap.4.0.1b.pl line 1638.
    Illegal division by zero at ..\heatmap.4.0.1b.pl line 1700.

  19. Pingback: penguinpunk.net » EMC – DIY Heatmaps – Updated Version

  20. Hey Scott,

    I’ve updated the script, the new version should update your database and fix the problem that you were seeing. The script should update your database file, and rename the data_al table

    Let me know if you continue to have problems

    thanks

    Mat.

  22. Pingback: penguinpunk.net » OT – 2013 and so on

  23. Pingback: penguinpunk.net » EMC – Some new scripts

  24. Pingback: penguinpunk.net » EMC – LUN Allocation Owner and Default Owner

  25. Mat,
    Would you be interested in perf data from a VNX5400?

    -Scott

  26. Hey Scott,

    that could be useful, I’ve been spending my time lately re-working the script, its now a client/server app, with a bunch of other graphing functionality. I hopefully should have a very alpha version available in the next few weeks hopefully. I’ve had a good range of NAR files to test with it, but its always good to get something different again, so yes … if you would be able to e-mail them to me (matharvest@gmail.com) that would be great

    thanks

    Mat

  27. Pingback: penguinpunk.net » EMC – HeatMap Analyzer v0.1 Now Available

  28. Wondering if there mightbe a problem with the download links for the new analyzer. All 3 links point to the same file:
    heatmap-analyzer_v0.1.tar.gz
    Doesn’t seem to be an appliance or documentation in there…

  30. Hi,
    i have problem with your script every time when i upload nar files i have this messages:
    Sun Aug 24 00:11:13 2014 /var/www/cgi-bin/processNAR.pl: Adding file: /heatmap_analyzer/tmp/nar/vnx.nar
    Sun Aug 24 00:11:13 2014 /var/www/cgi-bin/processNAR.pl: file: /heatmap_analyzer/tmp/nar/vnx.nar
    Sun Aug 24 00:11:13 2014 /var/www/cgi-bin/processNAR.pl: invalid Serial Number:

  31. Hi,

    Sorry to hear that you are having an issue with the script, hopefully we can sort out what is going on.

    I am happy for you to e-mail me directly at matharvest@gmail.com if you don’t want to post details in the comments section.

    Could you tell me the type/model of array, version of flarecode, and serial number and name of the array, and possibly run the following command over one of the NAR files and send me the output.

    naviseccli analyzer -archivedump -config -object stor

    I am guessing that you have an unusual character in your serial number or array name, its current restricted by the following regular expression [a-zA-Z0-9_-] (lines 319 & 349) of the processNAR.pl script.

    thanks

    Mat.

  32. How are you logging into the Data Domain with perl? I’m using Windows, and I can’t seem to login properly.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.