zenpacks on Andrés Álvarez

Extending Device Python Classes From Other ZenPacks

Tue, 27 Jun 2017 00:00:00 +0000

Recently I was struggling with trying to make two custom ZenPacks work together with device objects in the Zenoss database. My issue was that a main ZenPack (which would be installed first) would create a custom class that inherits from Device. This ZenPack would add a lot of properties and relationships with custom components. Then, a second optional ZenPack could be installed which would extend this same class with an additional property that would work along with a Python data source.

The zenpack.yaml file for the first and main ZenPack would look something like this:

# ZenPack1

classes:
  SpecialServer:
    base: [zenpacklib.Device]
    label: Special Server

  SpecialComponent:
    base: [zenpacklib.Component]
    label: Special Component
    properties:
      # ...

  AmazingComponent:
    base: [zenpacklib.Component]
    label: Amazing Component
    properties:
      # ...

class_relationships:
  - SpecialServer 1:MC SpecialComponent
  - SpecialServer 1:MC AmazingComponent

Overriding Default Zenoss Pages

Fri, 17 Mar 2017 00:00:00 +0000

While testing the Layer 2 ZenPack, I noticed that the network map provided by the ZenPack was placed in the same page of the old default Zenoss's network map, replacing the latter which was made using Flash. After the ZenPack is installed, when you click on the Network Map secondary link in the Infrastructure page, you will get the new map instead.

This seemed pretty nice, and I was curious how they achieved this, so I started browsing around the source code. It turns out that this overriding is done in a file called overrides.zcml, which is placed in the ZenPack's top directory.

Working With Zenoss Python Data Source Plugins

Thu, 23 Feb 2017 00:00:00 +0000

Using Python data source plugins in Zenoss is a great way to collect data, and probably a better way than using command datasources. Python data sources come with the introduction of the PythonCollector ZenPack, so this ZenPack is required in order to start using Python data sources in our own ZenPacks.

Python data source plugins work exceptionally great in replacing data collection logic in custom daemons written in Python. This means that the ZenPacks's code is greatly reduced because we do not have to create configuration service, and custom daemon code.

Moreover, while Python data source plugins are Python-based, we can still execute shell commands within the plugin.

Example Scenario: BMC Power Status

For this post, I will be using an example where we will be executing an ipmitool command to check the power chassis status of a BMC device.

If I run the following command from my shell:

ipmitool -H $BMC_IP -I lanplus -U admin -P admin power status

I get the following output:

Chassis Power is on

What we want is our data source plugin to periodically and continuously execute this command and obtain the value. Our ZenPack will then proceed to:

Map the value into a boolean property of a custon zenpacklib class.
Create a CRITICAL event if the power status is off or if the command fails. Create a CLEAR event if the power status is on.
Update the device model with the new property value.
Display the property's value in the device detail bar using JavaScript.

Processing Nova Live Migration Events in Zenoss

Mon, 23 Jan 2017 00:00:00 +0000

When monitoring OpenStack using the OpenStack Infrastructure ZenPack integrated with Ceilometer, it is possible to get fast virtual machine state changes (on/off powering, etc.) by receiving events sent from Ceilometer to Zenoss.

However I discovered that when trying to get the same effect for live migration (live migrating a virtual machine from one compute node to another) scenarios, this would not work. I proceeded to investigate why.

Ceilometer Dispatcher Live Migration Events

I decided that the first thing to check was if the Zenoss Ceilometer dispatcher was capturing and sending the live migration events to Zenoss. Indeed, the logs can be found in Ceilometer under /var/log/ceilometer/ceilometer-collector.log:

2017-01-23 14:22:02.593 25667 INFO ceilometer_zenoss.dispatcher.zenoss [-] record_events called (events=[<Event: 3538a01c-ab8d-4e64-b0ff-6e8fe270e06a, compute.instance.live_migration.post.dest.start, 2017-01-23 06:22:02.584049, <Trait: state_description 1 migrating> <Trait: memory_mb 2 512> <Trait: ephemeral_gb 2 0> <Trait: fixed_ips 1 [{u'version': 4, u'vif_mac': u'fa:16:3e:6e:02:e9', u'floating_ips': [], u'label': u'admin-net', u'meta': {}, u'address': u'192.168.0.15', u'type': u'fixed'}]> <Trait: user_id 1 6d581c230c86475abf70cce41440e8a1> <Trait: service 1 compute> <Trait: priority 1 info> <Trait: state 1 active> <Trait: launched_at 4 2017-01-23 05:06:41> <Trait: flavor_name 1 m1.tiny> <Trait: disk_gb 2 1> <Trait: display_name 1 pdcmtest> <Trait: root_gb 2 1> <Trait: tenant_id 1 10296907e44248d2a707689f77d59ef6> <Trait: instance_id 1 87be4b45-e214-4ca3-8f5c-1bd31159f9e4> <Trait: vcpus 2 1> <Trait: host_name 1 ndc27-3222> <Trait: request_id 1 req-d4c27d06-6329-4ce0-adb5-370b8ca83a22>>])

2017-01-23 14:22:02.771 25667 INFO ceilometer_zenoss.dispatcher.zenoss [-] record_events called (events=[<Event: 8db64329-606c-4184-9b5b-eb815166cb17, compute.instance.live_migration.post.dest.end, 2017-01-23 06:22:02.761138, <Trait: state_description 1 > <Trait: memory_mb 2 512> <Trait: ephemeral_gb 2 0> <Trait: fixed_ips 1 [{u'version': 4, u'vif_mac': u'fa:16:3e:6e:02:e9', u'floating_ips': [], u'label': u'admin-net', u'meta': {}, u'address': u'192.168.0.15', u'type': u'fixed'}]> <Trait: user_id 1 6d581c230c86475abf70cce41440e8a1> <Trait: service 1 compute> <Trait: priority 1 info> <Trait: state 1 active> <Trait: launched_at 4 2017-01-23 05:06:41> <Trait: flavor_name 1 m1.tiny> <Trait: disk_gb 2 1> <Trait: display_name 1 pdcmtest> <Trait: root_gb 2 1> <Trait: tenant_id 1 10296907e44248d2a707689f77d59ef6> <Trait: instance_id 1 87be4b45-e214-4ca3-8f5c-1bd31159f9e4> <Trait: vcpus 2 1> <Trait: host_name 1 ndc27-3222> <Trait: request_id 1 req-d4c27d06-6329-4ce0-adb5-370b8ca83a22>>])

2017-01-23 14:22:02.782 25667 INFO ceilometer_zenoss.dispatcher.zenoss [-] record_events called (events=[<Event: d0dc4f4a-454a-42c5-b767-386dc3e0d1f3, compute.instance.live_migration._post.end, 2017-01-23 06:22:02.776665, <Trait: state_description 1 migrating> <Trait: memory_mb 2 512> <Trait: ephemeral_gb 2 0> <Trait: fixed_ips 1 [{u'version': 4, u'vif_mac': u'fa:16:3e:6e:02:e9', u'floating_ips': [], u'label': u'admin-net', u'meta': {}, u'address': u'192.168.0.15', u'type': u'fixed'}]> <Trait: user_id 1 6d581c230c86475abf70cce41440e8a1> <Trait: service 1 compute> <Trait: priority 1 info> <Trait: state 1 active> <Trait: launched_at 4 2017-01-23 05:06:41> <Trait: flavor_name 1 m1.tiny> <Trait: disk_gb 2 1> <Trait: display_name 1 pdcmtest> <Trait: root_gb 2 1> <Trait: tenant_id 1 10296907e44248d2a707689f77d59ef6> <Trait: instance_id 1 87be4b45-e214-4ca3-8f5c-1bd31159f9e4> <Trait: vcpus 2 1> <Trait: host_name 1 ndc27-3205> <Trait: request_id 1 req-d4c27d06-6329-4ce0-adb5-370b8ca83a22>>])

Notice the live_migration.post.dest.start and live_migration.post.dest.end logs.

Appending Modeler Plugins in ZenPacks

Fri, 20 Jan 2017 00:00:00 +0000

When creating ZenPacks using zenpacklib, assigning modeler plugins to device classes will cause all the device class's modeler plugins to be replaced by the new modeler plugins assigned by the ZenPack. Obviously most of the time this is not the behaviour we want. What we really want is that the new modeler plugins are simply added to the device class's list of modeler plugins.

In a previous post I explained how we can customize the ZenPacks installation process to perform additional tasks. The way to append new modeler plugins to a device class follows this concept.

Let's assume that we have the OpenStack Infrastructure ZenPack installed in our Zenoss Core. This ZenPack adds some custom modeler plugins to the /Server/SSH/Linux/NovaHost device class. We want our custom ZenPack to add new modeler plugins to this device class without replacing the ones added by the OpenStack ZenPack.

This will be done upon installation (like in the previous post) inside the __init__.py file in the ZenPack top directory. This is how the code looks like:

Customizing the ZenPack Installation Process

Tue, 17 Jan 2017 00:00:00 +0000

There are some circumstances where we need to perform certain tasks in our ZenPack the moment it is installed. We can achieve this using Python by placing this logic inside the ZenPack's top directory's __init__.py file. When creating a fresh ZenPack using zenpacklib, this __init__.py file will contain the following contents:

from . import zenpacklib

CFG = zenpacklib.load_yaml()

To add custom functionality that gets executed when the ZenPack is installed, we need to extend the install method of the ZenPack class. Below the original code, we can proceed to do so:

class ZenPack(schema.ZenPack):
    def install(self, app):
        # Our custom logic here

        super(ZenPack, self).install(app)

No other imports are necessary. Notice the last line of the install method, this is where the ZenPack gets installed.

For the purpuse of giving an example, Let's say that our ZenPack creates two new zProperties, one property to store an API key, and the other one to store a URL. Moreover let's say that we obtain these values from somewhere and we want to assign them automatically to the properties upon the installation of the ZenPack.

These two new properties are defined in zenpack.yaml:

name: ZenPacks.aalvarez.MyZenPack

zProperties:
  zMyApiUrl:
    category: MyApi
    type: string

  zMyApiKey:
    category: MyApi
    type: string

Custom Pages in Zenoss

Mon, 26 Dec 2016 00:00:00 +0000

In a previous post I talked about how to create custom navigation links in Zenoss using a ZenPack. However we didn't get to creating the custom pages that these links would link to. This is what we will learn today.

Page Viewlets

Previously we created navigation item viewlets, this time we will be creating page viewlets. These type of viewlets allow us to create and insert new custom pages into Zenoss where we can display custom content. To create a new page, we add the following code to browser/configure.zcml:

<browser:page
  name="secondaryPage"
  for="*"
  permission="zenoss.View"
  template="templates/my_template.pt"
  />

It is important that the name attribute matches the URL of the navigation link we created previously. Another important attribute is the template attribute, this is a special file that will represent the base markup of the page using a combination of HTML, TAL, and METAL.

Also let's not forget to create a necessary python init file:

touch browser/__init__.py

Extending Zenoss Navigation Bars

Wed, 21 Dec 2016 00:00:00 +0000

It is possible to extend the functionality of Zenoss's navigations from within our custom ZenPacks. This means that we can add or remove links to the navigation bars we frequently use to access the infrastructure page or event console.

configure.zcml

Zenoss ZenPacks can contain a file in the ZenPack top directory called configure.zcml. I've mentioned and talked about this particular file in previous posts. This file basically acts as a configuration glue between back-end functions and front-end components.

It is in this file where we will declare and create new navigational links from our ZenPack.

~> In this post we assume that the ZenPack is created using zenpacklib, which is a Python library that makes creating ZenPacks much easier. Zenpacklib also makes the integration between the back-end and front-end much easier as well.

Usually this file begins with some required boilerplate code:

configure.zcml:

<?xml version="1.0" encoding="utf-8"?>
<configure
  xmlns="http://namespaces.zope.org/zope"
  xmlns:browser="http://namespaces.zope.org/browser"
  xmlns:zcml="http://namespaces.zope.org/zcml">

  <!-- Our custom code here -->
  <include package=".browser" />
</configure>

Custom Zenoss API Endpoints

Tue, 20 Dec 2016 00:00:00 +0000

In a previous post I went over the Zenoss JSON API and how it works inside the Zenoss Core. In this post we will apply the concepts learned in that post in order to create custom API endpoints within Zenoss, which can be accessed by the JavaScript front-end, curl, API clients, etc. All this new functionality added from a basic ZenPack.

Creating an Endpoint

Assuming that we are starting with a new and fresh ZenPack created with zenpacklib, we will proceed to create a simple endpoint.

Let's go ahead and create a file named api.py (can be any name) under the ZenPack's top directory. In this file we will import necessary modules, implement interfaces, and define our routers and facades. If you still don't know what routers and facades are and what they do, I suggest you first take a look at The Zenoss JSON API post.

First, let's take a quick look at the imports:

import os.path
from urlparse import urlparse
import subprocess

from zope.event import notify
from zope.interface import implements
from ZODB.transact import transact

from Products.ZenUtils.Ext import DirectRouter, DirectResponse
from Products import Zuul
from Products.Zuul.catalog.events import IndexingEvent
from Products.Zuul.facades import ZuulFacade
from Products.Zuul.interfaces import IFacade
from Products.Zuul.utils import ZuulMessageFactory as _t
from Products.ZenUtils.Utils import zenPath

Some imports are probably not needed, but the ones that are of our interest are imports such as DirectRouter and DirectResponse, which you might remember from the previous post. Additionally we are also importing the necessary facade, events, and interfaces imports.

SSH Monitoring in ZenPacks

Wed, 14 Dec 2016 00:00:00 +0000

Zenoss usually monitors and collects information using SNMP and SSH methods. A good example of is the linux monitor ZenPack which can collect information such as hard disks, interfaces, and file systems using both methods.

SNMP works by installing and configuring an SNMP agent on the machine we want to monitor. This agent will poll the machine for data, and this data can be retrieved by Zenoss using net-snmp.

On the other hand, SSH works by configuring a username and password or a path to an SSH key zProperties. This will allow Zenoss to remotely access the host using SSH and execute the corresponding commands and return the corresponding information.

When developing ZenPacks, data collection using SNMP is a relatively common practice. There is even a very good tutorial on SNMP monitoring using zenpacklib. When using SNMP, the modeler plugins are SnmpPlugin plugins that typically work using a set of OIDs to determine exactly which data is to be modeled.

SSH in Modeler Plugins

In order to make a modeler plugin utilize SSH as it's polling method we need to use a different type of plugin, the CommandPlugin plugin. The structure of this type of plugin is very similar to the SnmpPlugin, however there is one critical variable that must be implemented.

Modifying the Zenoss Infrastructure Grid

Thu, 10 Nov 2016 00:00:00 +0000

Continuing from my previous post, where I explained how to modify the device detail bar from a ZenPack using ExtJS, in this post I will explain how we can modify the device list grid shown in the infrastructure page.

The Infrastructure Grid

This is the table in the infrastructure page that shows all devices being monitored by Zenoss. Default columns include device name, device class, IP address, production state, and events, as shown in the image below:

However it would be nice to also include the power status we added to the device detail bar in the previous post, maybe even add the ping status as well (why Zenoss doesn't do this by default is beyond me).

DevicePanels.js

The source code for the grid is found in $ZENHOME/Products/ZenUI/browser/resources/js/zenoss/DevicePanels.js. A quick glance at it and you will quickly find the definitions of the columns I mentioned earlier, defined in an array called deviceColumns.

Modifying the Zenoss Device Detail Bar

Tue, 08 Nov 2016 00:00:00 +0000

The Zenoss Core ExtJS graphic user interface is divided into many different components. In this post I will go over on how we can modify the device detail bar to display additional custom information for devices, by creating a custom ZenPack.

Zenoss Core 4 uses ExtJS 4 JavaScript framework to manage all the user interface components. These components can be found in $ZENHOME/Products/ZenUI3/browser/resources/js/zenoss, and as you would expect, the device detail bar component is also located there.

The Device Detail Bar

This is the detail bar located on the device view page which shows the device's icon, name, events, status, production state, and priority. The JavaScript source code for this component can be found in DeviceDetailBar.js

However since we are going to extend this component through a ZenPack, we will not modify that source. Instead, we will add new code to use ExtJS to add our custom data displays.

Extending The Component, Through ZenPacks

Let's assume we are starting with a freshly created ZenPack, which adds a new power_status integer field to certain devices, according to its zenpack.yaml file:

# ...

classes:
  CustomDevice:
    base: [zenpacklib.Device]
    label: Custom Device
    properties:
      power_status:
        type: boolean

What we want is to display this value in the device's detail bar, similar to the device's ping status display.

Creating Zenoss ZenPack Daemons

Mon, 07 Nov 2016 00:00:00 +0000

ZenPacks are powerful custom add-ons that can help us extend Zenoss's functionality. In this post I will go over on how to create a ZenPack that adds a custom daemon to the existing Zenoss daemons and runs on a configured cycle time to perform custom tasks. We will achieve this by creating the ZenPack using zenpacklib, but it is also possible to create it from the Zenoss user interface.

About zenpacklib

zenpacklib is a Python library developed by the Zenoss team to facilitate the process of creating ZenPacks, specially ZenPacks that deal with modeling and monitoring devices and components. Most of the newer ZenPacks that are being released are now being built with zenpacklib.

We can obtain zenpacklib by running the following commands:

wget http://zenpacklib.zenoss.com/zenpacklib.py
chmod 755 zenpacklib.py

This will download the zenpacklib Python library and give it executable permissions.

Creating the ZenPack

Using the zenpacklib file we just downloaded, we proceed to create a new fresh ZenPack:

./zenpacklib.py create ZenPacks.<your_namespace>.<zenpack_name>

This will create the ZenPack base directory with the necessary base files.

Now we go into this directory to begin.

cd ZenPacks.<your_namespace>.<zenpack_name>

Creating the ZenPack Daemon

Our custom daemon declaration will be located in a directory named daemons, inside our ZenPack directory:

mkdir ZenPacks.<your_namespace>.<zenpack_name>/<your_namespace>/<zenpack_name>/daemons

Here we will create a new Bashscript file with the name of our daemon. In this case we will name it mydaemon:

#! /usr/bin/env bash

DAEMON_NAME="mydaemon"

. $ZENHOME/bin/zenfunctions

MYPATH=`python -c "import os.path; print os.path.realpath('$0')"` THISDIR=`dirname $MYPATH` PRGHOME=`dirname $THISDIR` PRGNAME=$DAEMON_NAME.py CFGFILE=$CFGDIR/$DAEMON_NAME.conf

generic "$@"

Once the ZenPack is installed, files under this daemons directory will become executable (chmod 0755), a symlink to the file will be created in $ZENHOME/bin, and a configuration file will be generated in $ZENHOME/etc/<daemon_name>.conf

NOTE: If you created your ZenPack using the Zenoss user interface, the daemons directory will also be automatically created, and will contain an example daemon file named zenexample with code similar to the one above. In this case you should simply replace the necessary values.

Zenoss Renderers

Thu, 27 Oct 2016 00:00:00 +0000

Zenoss makes use of some very interesting graphical components called renderers. These are used to manipulate the way data is shown in the Zenoss user interface.

For example, a value of total bytes used could be 6080626688 in bytes, which is a very high number and doesn't really convey much meaning. However, we can use a built-in Zenoss renderer called bytesString which will convert this value in bytes to the closest representation:

As we can see, this is a much better and meaningful way of displaying the data.

The built-in Zenoss renderers and source code can be found in $ZENHOME/Products/ZenUI3/browser/resources/js/zenoss/Renderers.js

They are a list of registered Javascript functions that can be assigned in our YAML definitions file by adding the renderer property. For example:

HardDisk:
  base: [zenpacklib.Component]
  label: Hard Disk
  properties:
     location:
        label: Location

     capacity:
        label: Capacity

     raid_name:
        label: Raid Name

     raid_level:
        label: Raid Level

     status:
        label: Status
        renderer: Zenoss.render.pingStatus

Zenoss.render.pingStatus is one of the default renderers that come built-in within Zenoss Core 4, similar to Zenoss.render.bytesString.

Changing Zenoss Dashboard Portlets Using CLI

Wed, 26 Oct 2016 00:00:00 +0000

The Zenoss ZenPacks.zenoss.Dashboard ZenPack adds a very nice and new dashboard to our Zenoss Core 4 deployment, replacing the old default dashboard that comes with the installation.