python on Andrés Álvarez

Implementing Rate Limiting in Chalice

Mon, 23 Jan 2023 00:00:00 +0000

Chalice (AWS's Python serverless micro-framework) does not offer built-in support for throttling/rate limiting. In this post I will show you how to implement your own custom rate limiting capabilities for your serverless API.

Setting Up a SQLAlchemy and Pytest Based Test Suite

Wed, 25 May 2022 00:00:00 +0000

✨ This is an improved article of the previous Python Transactional Tests Using SQL Alchemy, Pytest, and Factory Boy article.

In this post I will show you how you can setup a Pytest test suite that automatically sets up and teardowns your database using SQAlchemy on each test run. It will also provide transactional tests capabilities as described in Python Transactional Tests Using SQL Alchemy, Pytest, and Factory Boy.

How to Test Amazon SES Emails Using Pytest and Moto

Fri, 30 Jul 2021 09:58:59 -0600

In this post I will show you a very neat way to unit test your Amazon SES emails using Pytest and the awesome Moto library for mocking AWS services.

How I Define SES Emails in Application Code

I like to use a handy base class for preparing and sending SES emails within my application code. Here is a simple example:

Python Transactional Tests Using SQL Alchemy, Pytest, and Factory Boy

Fri, 19 Feb 2021 13:59:36 -0600

🆕 Read the newer and improved post Setting Up a SQLAlchemy and Pytest Based Test Suite

📅 Last Updated on November 26 2021

Knowing how to integrate SQLAlchemy for your automated tests can be a bit tricky. Even after reading several blog posts on how to implement transactional tests with SQLAlchemy 1.3 and Pytest, it took me a bit of time to make it all play nicely with Factory Boy.

In this post I will teach you how you that is achieved.

How to Debug a Chalice App in Vscode

Thu, 24 Dec 2020 12:53:55 -0600

In this post I will show you how you can create a debug configuration in Visual Studio Code that will allow you to debug a Chalice application locally.

Gracefully Exiting Python Context Managers on Ctrl+C

Fri, 10 Apr 2020 16:27:41 -0600

In this post I will show you how you can exit gracefully from within a context manager in your Python application.

Suppose that we provide a context manager that can be used as a session to perform certain tasks. When the context manager is closed, there is some cleanup work to be done. We want this to happen even when the user interrupts the program with Ctrl+C key.

This is how we can achieve that:

Python Log Messages With Dynamic Formatter Variables

Thu, 02 Apr 2020 16:07:28 -0600

In this post I will explain how it is possible to replace logger formatter variables with dynamic values. Take the following log message for example:

[2020-04-02 15:37:01] myapp INFO: Logged in successfully.

What if we wanted to add some additional data after the INFO part, but this data is dynamic, or is only available at runtime. Like a username or an app name for example.

Let's take a look at how to achieve this.

Making Functions Timeout in Python

Wed, 06 Feb 2019 16:02:35 +0000

Sometimes you need to execute a function that can take a lot of time to finish. You are not sure exactly when the function will finish, but you do not want to wait too long, or let your program “hang” waiting for a response.

We want our function to run for a certain period of time, and if this time limit is exceeded, we want to regain control of the program's execution.

We can achieve this by using a custom context manager and the signal module from the Python standard library.

Here is a complete example:

Python Functional Testing for AWS Lambda

Wed, 26 Dec 2018 17:17:28 +0000

In this post I will explain and go over how I personally write functional tests for my AWS Lambda functions.

For our example use case, we will assume that our Lambda function is quite complex, as in that it involves downloading a file, processing it, and uploading a result. Additionally it will be called by an SQS event, and it make use of other non-Python related elements, as we will see later.

Let's get started!

Pytest Tricks for Better Python Tests

Fri, 21 Dec 2018 21:03:28 +0000

Pytest is my Python testing framework of choice. Very easy to use, and makes tests look much better.

In this article I'll show you some cool tricks I have incorporated into my test suites using Pytest.

Environment Variables

When your application must work with defined environment variables, the testing environment must have these variables defined as well, even if the values are not real.

With Pytest we can easily configure any necessary environment variables in our test environment. We simply create a fixture in conftest.py that will be loaded automatically:

# conftest.py

import pytest


@pytest.fixture(autouse=True)
def env_setup(monkeypatch):
    monkeypatch.setenv('MY_SETTING', 'some-value')
    monkeypatch.setenv('ANOTHER_SETTING', 'some-value')

monkeypatch is a built-in pytest fixture that allows us to set environment variables in the test runs. By enabling the autouse option, our custom environment setup fixture will be automatically called in every test without having to include it explicitly using the usual dependency injection mechanism.

Recursive Python AWS Lambda Functions

Tue, 18 Sep 2018 00:00:00 +0000

There are times where some processing task cannot be completed under the AWS Lambda timeout limit (a maximum of 5 minutes as of this writing). A possible solution for these kind of situations is to implement a recursive approach to perform the processing task.

Basically, if you are able to separate the task into multiple chunks or batches, then you could make each batch to be processed by a different Lambda function. The amount of Lambda functions necessary to finish the task will scale accordingly.

BytesIO File Uploads to Django Using Requests

Tue, 26 Jun 2018 00:00:00 +0000

It's very easy to post file data to Django using requests:

import requests


requests.post(url, files={'cover': open('imgpath.jpg', 'rb')})

However I was having a hard time getting that to work using BytesIO. The reason I wanted to use BytesIO was because I was reading the file binary data located in S3, from AWS Lambda. I didn't want to write the file to disk first and then do something like the code shown above.

Here's how to achieve that:

Testing Django and DRF With Pytest

Fri, 09 Feb 2018 00:00:00 +0000

Pytest has become my favorite Python testing framework. And in this article I want to go over on how I learned to write nice tests for Django and Django REST Framework.

We will be using the following tools:

Pytest: Python testing framework
pytest-django: Pytest extensions for Django
factoryboy: Factories for easy test data generation.

Django AJAX Search With Elasticsearch

Wed, 10 Jan 2018 00:00:00 +0000

In this post I want to go over how I implemented a very neat AJAX search using Elasticsearch in Django. Let's get started!

The Search Template

The starting point of our search feature will be the search template. That is, the template that contains the input field where the user will type in the search keywords.

This template is extremely simple and does not need a <form> tag. I will create it as a partial under templates/search.html:

{% csrf_token %}
<input id="search" name="q" placeholder="Search...">

Notice how we have given the input a name with the value of q. This will be important when our view receives the query from this input field.

I can then render this partial in the main template like this:

{% include 'search.html' %}

Bulk Create With Django REST Framework

Mon, 27 Nov 2017 00:00:00 +0000

In this quick post I will go over how to create an endpoint that allows bulk creation of a resource using Django REST Framework.

The Serializer

Let's assume that we will be working with an existing Book model in our Django application. We will create a serializer for this model using Django REST's ModelSerializer helper class:

# serializers.py
from myapp.models import Book
from rest_framework import serializers


class BookSerializer(serializers.ModelSerializer):
    class Meta:
        model = Book
        fields = '__all__'

The "__all__" string indicates that we want to serialize all of the model's fields.

Extending Device Python Classes From Other ZenPacks

Tue, 27 Jun 2017 00:00:00 +0000

Recently I was struggling with trying to make two custom ZenPacks work together with device objects in the Zenoss database. My issue was that a main ZenPack (which would be installed first) would create a custom class that inherits from Device. This ZenPack would add a lot of properties and relationships with custom components. Then, a second optional ZenPack could be installed which would extend this same class with an additional property that would work along with a Python data source.

The zenpack.yaml file for the first and main ZenPack would look something like this:

# ZenPack1

classes:
  SpecialServer:
    base: [zenpacklib.Device]
    label: Special Server

  SpecialComponent:
    base: [zenpacklib.Component]
    label: Special Component
    properties:
      # ...

  AmazingComponent:
    base: [zenpacklib.Component]
    label: Amazing Component
    properties:
      # ...

class_relationships:
  - SpecialServer 1:MC SpecialComponent
  - SpecialServer 1:MC AmazingComponent

Zenoss Custom Notification Actions Using Subscribers

Wed, 07 Jun 2017 00:00:00 +0000

In a previous post I talked about how we could create custom notifications actions in Zenoss. Like a SMS notification, for example. In that post we required a valid cellphone number using a text field in the notification's content pane, as shown below:

In this post I want to change and improve this by using subscribers functionality instead. Similar to the e-mail (or pager) notification, which sends an e-mail to all the subscribed users using their e-mail address configured in their user settings. Since there is no SMS mobile number that we can configure in the user settings, we will have to use the Pager field instead.

Using Zenoss Core 4's pager action's source code as reference is a good starting point, since we are going to use the pager field for SMS.

We can see that this action class implements a executeOnTarget method instead of a execute method we used in the previous post.

Python Functional HTTP Testing With Gabbi

Tue, 06 Jun 2017 00:00:00 +0000

Continuing with my journey in contributing to Gnocchi, I have learned of an excellent tool which Gnocchi uses to test its API from a HTTP request approach. The tool is called Gabbi.

Coming from a Ruby, Rails, and Rspec background, I was very pleased on learning how Gabbi works and how to use it. Gabbi uses YAML to construct the tests that will hit the API endpoints, in a similar way to using Rspec's DSL when making request specs.

For my latest contribution to Gnocchi, I was working on an issue where newly created metrics were not being returned in the response from /v1/resource/generic/:id/metric. After applying the fixes and submitting the patch, it was also necessary to update some of the functional Gabbi tests that belong to this specific use case.

One of these original tests is shown below:

Understanding Gnocchi Measures

Wed, 24 May 2017 00:00:00 +0000

Measures are simple objects that represent the timeseries data. They simply contain a timestamp and a value, and they belong to metrics. You could say that a measure has many metrics.

Adding Measures to a Metric

Using the REST API, we can easily add measures to a metric:

POST /v1/metric/511b61a1-8d67-42d5-8add-66d4209a0469/measures HTTP/1.1
Content-Type: application/json
Content-Length: 198

Understanding Gnocchi Metrics

Thu, 18 May 2017 00:00:00 +0000

Metrics are one of the main object types in Gnocchi. They are identified by a UUID and they can also be attached to a resource by using a resource name. Metrics store measures, and the way they do this is defined by archive policies. These are concepts that I will cover in future articles.

Basically, a metric designates any thing that can be measured: the CPU usage of a server, the temperature of a room or the number of bytes sent by a network interface.

In the Gnocchi architecture, the storage back-end is responsible for storing measures of created metrics. It receives timestamps and values, and pre-computes aggregations according to the defined archive policies.

Custom Notification Action Types in Zenoss

Tue, 09 May 2017 00:00:00 +0000

In a previous post I talked about how we could create command type notifications to execute shell scripts when a trigger is fired by an event in Zenoss. Also, in a more recent post I explained how a ZenPack could automatically create these triggers and notifications when installed by defining them in JSON format.

In this post I will explain how to create additional custom action types that can be used by notifications for different purposes.

Zenoss Default Notification Actions

By default, Zenoss comes with a built-in notification action types. You can see them when creating a new notification from the user interface:

These action types are:

Command: Runs an executable script. Discussed in one of my previous posts.
Email: Sends an e-mail to a user.
Page: Sends a message to a user's pager, also known as beeper. Probably safe to say that this is not used by anyone anymore.
Syslog: Logs a message to syslog
SNMP Trap: Sends an SNMP trap.

All these notifications are sent when an event matches a trigger rule.

Creating the New Action

Next, we are going to learn how we can create and add a new fully functional custom action to this list, using a ZenPack. For this example, let's assume that we want to create a new SMS action to send SMS messages. Surely something more modern and useful than a Pager!

Custom Triggers and Notifications in a ZenPack

Thu, 04 May 2017 00:00:00 +0000

In a previous post I talked about how to use triggers and command notifications in Zenoss to trigger custom actions when certain events occur. All of this was done using the Zenoss user interface.

In this post we will achieve something similar, but from a custom ZenPack. At the end of this post, our custom ZenPack will be able to create new custom triggers and notifications when installed.

Defining ZenPack Triggers & Notifications

Triggers and notifications within a ZenPack are actually defined using JSON. To do so, you must create a file named actions.json in a directory called zep (create it if it doesn't exist), within the ZenPack's top directory.

actions.json:

{
  "triggers": [
    {
      "name": "Critical_death_event",
      "uuid": "4c055067-98b7-483e-8f49-2820b4f2f721",
      "enabled": false,
      "rule": {
        "api_version": 1,
        "type": 1,
        "source": "(evt.event_class.startswith(\"/Status/Ping\")) and (evt.status == 0) and (evt.severity > 2)"
      }
    }
  ],

  "notifications": [
    {
      "id": "send_sms_message",
      "description": "Send SMS using Twilio",
      "action": "command",
      "guid": "2606439f-5ef7-40dc-90e4-3f3bee11cfe6",
      "enabled": false,
      "action_timeout": 60,
      "delay_seconds": 0,
      "repeat_seconds": 0,
      "send_initial_occurrence": false,
      "send_clear": false,
      "body_format": "echo \"Hello World!\"",
      "clear_body_format": "",
      "subscriptions": ["4c055067-98b7-483e-8f49-2820b4f2f721"]
    }
  ]
}

Diving Into OpenStack Gnocchi

Tue, 25 Apr 2017 00:00:00 +0000

Gnocchi is a multi-tenant timeseries, metrics and resources database. It provides an HTTP REST interface to create and manipulate the data. It is designed to store metrics at a very large scale while providing access to metrics and resources information and history.

It is the preferred storage method for metrics in Ceilometer, as of OpenStack Ocata.

In this post I want to dive into Gnocchi specifics such as its configuration, supported backends, APIs, daemons, and source code.

Configuration

Gnocchi's configuration is stored in a file called gnocchi.conf. Ideally, this file would be in ~/gnocchi.conf or /etc/gnocchi/gnocchi.conf. Let's take a look at a basic Gnocchi configuration:

[DEFAULT]
debug = true
verbose = true

[api]
workers = 1

[database]
backend = sqlalchemy

[indexer]
url = postgresql://gnocchi:gnocchi@127.0.0.1/gnocchi

[storage]
coordination_url = file:///home/ubuntu/gn/locks
driver = file
file_basepath = /home/ubuntu/gn

[cors]
allowed_origin = *
allow_credentials = false

The configuration above sets up Gnocchi to use Postgresql as the indexer, and use the file system for storage. Additionally it sets up CORS so that requests from any origin are allowed. You will want to configure CORS in a more secure manner when deploying to a production environment.

Database Setup

For this example, we are going to use Cloud 9 as our environment, and Postgresql as the database. This means that we need to first setup the database before we start using Gnocchi.

Make sure the Postgresql service is running:

sudo service postgresql start

We can enter the Postgresql command line using:

sudo sudo -u postgres psql

Now let's create a new Postgresql user:

CREATE USER gnocchi SUPERUSER PASSWORD 'gnocchi';

Then create the database:

CREATE DATABASE gnocchi WITH TEMPLATE = template0 ENCODING = 'UNICODE';

When the database is finally set up correctly and the configuration file is in place, we can initialize the indexer and storage:

gnocchi-upgrade

You should see the following output logs:

2017-05-17 05:28:50.917 3895 INFO gnocchi.cli [-] Upgrading indexer <gnocchi.indexer.sqlalchemy.SQLAlchemyIndexer object at 0x7ff76cff6190>
2017-05-17 05:28:50.982 3895 INFO alembic.runtime.migration [-] Context impl PostgresqlImpl.
2017-05-17 05:28:50.982 3895 INFO alembic.runtime.migration [-] Will assume transactional DDL.
2017-05-17 05:28:51.011 3895 INFO alembic.runtime.migration [-] Context impl PostgresqlImpl.
2017-05-17 05:28:51.011 3895 INFO alembic.runtime.migration [-] Will assume transactional DDL.
2017-05-17 05:28:51.154 3895 INFO gnocchi.cli [-] Upgrading storage <gnocchi.storage.file.FileStorage object at 0x7ff7688f9710>

Gnocchi REST API

Gnocchi's REST API is based on Pecan, a very lightweight Python web framework that provides object-dispatch style routing. We can confirm this in Gnocchi's rest/__init__.py file:

import pecan
from pecan import rest

Metrics

Gnocchi provides an object type that is called metric. A metric designates any thing that can be measured: the CPU usage of a server, the temperature of a room or the number of bytes sent by a network interface.

A metric only has a few properties: a UUID to identify it, a name, the archive policy that will be used to store and aggregate the measures.

Farther down the code in rest/__init__.py, we can find a metric controller which inherits from a Pecan REST controller:

class MetricController(rest.RestController):
    _custom_actions = {
        'measures': ['POST', 'GET']
    }

    def __init__(self, metric):
        self.metric = metric
        mgr = extension.ExtensionManager(namespace='gnocchi.aggregates',
                                         invoke_on_load=True)
        self.custom_agg = dict((x.name, x.obj) for x in mgr)

    def enforce_metric(self, rule):
        enforce(rule, json.to_primitive(self.metric))

    @pecan.expose('json')
    def get_all(self):
        self.enforce_metric("get metric")
        return self.metric

From the Pecan documentation, we can learn that Pecan uses a routing strategy known as object-dispatch to map an HTTP request to a controller, and then the method to call. Object-dispatch begins by splitting the path into a list of components and then walking an object path, starting at the root controller.

We can tell Pecan which methods in a class are publically-visible via expose(). If a method is not decorated with expose(), Pecan will never route a request to it. In the example above, the get_all() method is exposed to Pecan. Additionally, it makes use of Pecan's built-in support for a special JSON renderer, which translates template namespaces into rendered JSON text. Meaning that the returned content will be rendered as JSON.

Understanding oslo_config in OpenStack

Tue, 18 Apr 2017 00:00:00 +0000

The OpenStack Oslo library provides a set of Python libraries containing code shared by different OpenStack projects. This helps the many different OpenStack projects to follow a certain standard and convention when being developed.

In this post I want to go over oslo_config. This library helps with parsing of options found in configuration files or command line arguments.

Understanding Options

An option is defined using the Opt class or one of its sub-classes, from the cfg module:

from oslo_config import cfg

common_opts = [
    cfg.StrOpt('bind_host',
               default='0.0.0.0',
               help='IP address to listen on.'),
    cfg.Opt('bind_port',
            type=PortType,
            default=9292,
            help='Port number to listen on.')
]

In the example above we can see that we are using Opt and StrOpt, one of its sub-classes. You might notice that the second option's constructor has a type=PortType argument. This argument is a callable object that takes a string and either returns a value of that particular type, or raises ValueError if the value cannot be converted. The different types available can be found in the types module:

Understanding Ceilometer Publishers

Tue, 11 Apr 2017 00:00:00 +0000

Continuing with my study of the Ceilometer pipeline, this post now covers Ceilometer publishers. Publishers are components that make it possible to save the data into a persistent storage through the message bus, or to send it to one or more external consumers.

Publishers are specified in the publishers section for each pipeline that is defined in the pipeline.yaml and the event_pipeline.yaml files.

Many different publishers are available. The following are the most common and important publishers:

Gnocchi

Understanding Ceilometer Transformers

Fri, 07 Apr 2017 00:00:00 +0000

Ceilometer transformers are part of the Ceilometer pipeline, which is the mechanism by which data is processed. In Ceilometer there is a pipeline for samples and a pipeline for events.

Configurations for these pipelines can be found in /etc/ceilometer/pipeline.yaml and /etc/ceilometer/event_pipeline.yaml.

Transformers are responsible for mutating data points and passing them to publishers that will send the data to external systems.

Different types of transformers exist to mutate different types of data. The following is a table containing all the available transformer types:

Name of transformer	Reference name for configuration
Accumulator	accumulator
Aggregator	aggregator
Arithmetic	arithmetic
Rate of change	rate_of_change
Unit conversion	unit_conversion
Delta	delta

Let's take a more detailed look into the accumulator transformer.

Setting Up an OpenStack Dev Environment

Wed, 22 Mar 2017 00:00:00 +0000

Before beginning to contribute to OpenStack, it is necessary that we setup an ideal development environment for a smoother workflow.

In this post I will cover how to setup a Devstack development environment in a Ubuntu 16.04 virtual machine using VMware Player.

Setting Up the Ubuntu Virtual Machine

Make sure you have VMware Player installed in your system. Then we need to choose an appropriate Ubuntu ISO image for our virtual machine. In my case I am using a Ubuntu Desktop 16.04.2 LTS (Xenial Xerus) for 32-bit PC i836.

In VMware Player, create a new virtual machine using the wizard. Assign your preferred amount of memory and storage space. Now start the machine, and select the Ubuntu .iso image file when prompted. Proceed with the installation and select the Erase Disk and Install Ubuntu option when prompted.

~> I tried using VirtualBox for virtualization but I could not successfully install Devstack with it. Probably due to issues on using the correct network adapter. I also tried using Ubuntu 14.04 without success as well. This is why I recommend Ubuntu 16.04.

Once the installation is finished, restart the virtual machine.

Getting Started on Contributing to OpenStack

Tue, 14 Mar 2017 00:00:00 +0000

I have started to become interested in contributing to the OpenStack project. Starting to contribute to such a massive project can seem like a daunting task, and it is. It is only by arming yourself with the proper tools and knowledge prior to contributing, that you will have a smoother process. I created this blog post to smooth that process.

Picking a Project

OpenStack is composed of many different projects for different purposes (compute, storage, networking, telemetry, etc.). Before you start contributing, you must have an idea of which project you actually want to contribute to. In my case I have decided on the Telemetry part, more specifically on Ceilometer.

After you have picked a project, it is time to start reading its documentation and developer documentation to understand how it works. The Ceilometer developer documentation offers good explanations and diagrams of its architecture and components.

Here is another nice guide on contributing to Ceilometer.

Python Tricks for Better Code

Wed, 08 Mar 2017 00:00:00 +0000

A compilation of very useful Python tricks for better code. All examples are Python 2.7 and Python 3 compatible unless stated otherwise.

Enumerate

When working with items in a list and we want to print or handle each item's index, instead of using a counter variable:

# Bad way
i = 0
for place in places:
    print(i, place)
    i += 1

We can use enumerate to improve our code:

for i, place in enumerate(places):
    print(i, place)

enumerate returns a list of indexes along with each item in the list.

Working With Zenoss Python Data Source Plugins

Thu, 23 Feb 2017 00:00:00 +0000

Using Python data source plugins in Zenoss is a great way to collect data, and probably a better way than using command datasources. Python data sources come with the introduction of the PythonCollector ZenPack, so this ZenPack is required in order to start using Python data sources in our own ZenPacks.

Python data source plugins work exceptionally great in replacing data collection logic in custom daemons written in Python. This means that the ZenPacks's code is greatly reduced because we do not have to create configuration service, and custom daemon code.

Moreover, while Python data source plugins are Python-based, we can still execute shell commands within the plugin.

Example Scenario: BMC Power Status

For this post, I will be using an example where we will be executing an ipmitool command to check the power chassis status of a BMC device.

If I run the following command from my shell:

ipmitool -H $BMC_IP -I lanplus -U admin -P admin power status

I get the following output:

Chassis Power is on

What we want is our data source plugin to periodically and continuously execute this command and obtain the value. Our ZenPack will then proceed to:

Map the value into a boolean property of a custon zenpacklib class.
Create a CRITICAL event if the power status is off or if the command fails. Create a CLEAR event if the power status is on.
Update the device model with the new property value.
Display the property's value in the device detail bar using JavaScript.

Useful Zendmd Tricks

Tue, 14 Feb 2017 00:00:00 +0000

The Zenoss Zendmd Tips Wiki page contains a few useful tricks using zendmd to perform tasks. In this post I am adding more tricks that I discover and learn along the way.

Removing Device Classes

We can easily remove device classes within zendmd with a simple command. Assuming we want to remove the default “KVM” device class:

dmd.Devices.manage_deleteOrganizer("/zport/dmd/Devices/KVM")
commit()

Change Zenoss User's Password

Let's say we want to change the default admin user's password (zenoss) in the Ubuntu auto deploy:

app.acl_users.userManager.updateUserPassword('admin', 'newpassword')
commit()

Appending Modeler Plugins in ZenPacks

Fri, 20 Jan 2017 00:00:00 +0000

When creating ZenPacks using zenpacklib, assigning modeler plugins to device classes will cause all the device class's modeler plugins to be replaced by the new modeler plugins assigned by the ZenPack. Obviously most of the time this is not the behaviour we want. What we really want is that the new modeler plugins are simply added to the device class's list of modeler plugins.

In a previous post I explained how we can customize the ZenPacks installation process to perform additional tasks. The way to append new modeler plugins to a device class follows this concept.

Let's assume that we have the OpenStack Infrastructure ZenPack installed in our Zenoss Core. This ZenPack adds some custom modeler plugins to the /Server/SSH/Linux/NovaHost device class. We want our custom ZenPack to add new modeler plugins to this device class without replacing the ones added by the OpenStack ZenPack.

This will be done upon installation (like in the previous post) inside the __init__.py file in the ZenPack top directory. This is how the code looks like:

Customizing the ZenPack Installation Process

Tue, 17 Jan 2017 00:00:00 +0000

There are some circumstances where we need to perform certain tasks in our ZenPack the moment it is installed. We can achieve this using Python by placing this logic inside the ZenPack's top directory's __init__.py file. When creating a fresh ZenPack using zenpacklib, this __init__.py file will contain the following contents:

from . import zenpacklib

CFG = zenpacklib.load_yaml()

To add custom functionality that gets executed when the ZenPack is installed, we need to extend the install method of the ZenPack class. Below the original code, we can proceed to do so:

class ZenPack(schema.ZenPack):
    def install(self, app):
        # Our custom logic here

        super(ZenPack, self).install(app)

No other imports are necessary. Notice the last line of the install method, this is where the ZenPack gets installed.

For the purpuse of giving an example, Let's say that our ZenPack creates two new zProperties, one property to store an API key, and the other one to store a URL. Moreover let's say that we obtain these values from somewhere and we want to assign them automatically to the properties upon the installation of the ZenPack.

These two new properties are defined in zenpack.yaml:

name: ZenPacks.aalvarez.MyZenPack

zProperties:
  zMyApiUrl:
    category: MyApi
    type: string

  zMyApiKey:
    category: MyApi
    type: string

Fixing Zenoss Device Network Interface Graphs

Fri, 13 Jan 2017 00:00:00 +0000

When testing the monitoring of the OpenStack compute node devices in Zenoss (using OpenStack Infrastructure ZenPack), I noticed that I could not get any graphs for the network interfaces of the device, although we could still get and model all the available interfaces:

This was indeed strange because I could perfectly get the graphs for other devices. After a lot of head scratching and prying around the code and interface, I finally found the reason, which I explain below.

Custom Zenoss API Endpoints

Tue, 20 Dec 2016 00:00:00 +0000

In a previous post I went over the Zenoss JSON API and how it works inside the Zenoss Core. In this post we will apply the concepts learned in that post in order to create custom API endpoints within Zenoss, which can be accessed by the JavaScript front-end, curl, API clients, etc. All this new functionality added from a basic ZenPack.

Creating an Endpoint

Assuming that we are starting with a new and fresh ZenPack created with zenpacklib, we will proceed to create a simple endpoint.

Let's go ahead and create a file named api.py (can be any name) under the ZenPack's top directory. In this file we will import necessary modules, implement interfaces, and define our routers and facades. If you still don't know what routers and facades are and what they do, I suggest you first take a look at The Zenoss JSON API post.

First, let's take a quick look at the imports:

import os.path
from urlparse import urlparse
import subprocess

from zope.event import notify
from zope.interface import implements
from ZODB.transact import transact

from Products.ZenUtils.Ext import DirectRouter, DirectResponse
from Products import Zuul
from Products.Zuul.catalog.events import IndexingEvent
from Products.Zuul.facades import ZuulFacade
from Products.Zuul.interfaces import IFacade
from Products.Zuul.utils import ZuulMessageFactory as _t
from Products.ZenUtils.Utils import zenPath

Some imports are probably not needed, but the ones that are of our interest are imports such as DirectRouter and DirectResponse, which you might remember from the previous post. Additionally we are also importing the necessary facade, events, and interfaces imports.

SSH Monitoring in ZenPacks

Wed, 14 Dec 2016 00:00:00 +0000

Zenoss usually monitors and collects information using SNMP and SSH methods. A good example of is the linux monitor ZenPack which can collect information such as hard disks, interfaces, and file systems using both methods.

SNMP works by installing and configuring an SNMP agent on the machine we want to monitor. This agent will poll the machine for data, and this data can be retrieved by Zenoss using net-snmp.

On the other hand, SSH works by configuring a username and password or a path to an SSH key zProperties. This will allow Zenoss to remotely access the host using SSH and execute the corresponding commands and return the corresponding information.

When developing ZenPacks, data collection using SNMP is a relatively common practice. There is even a very good tutorial on SNMP monitoring using zenpacklib. When using SNMP, the modeler plugins are SnmpPlugin plugins that typically work using a set of OIDs to determine exactly which data is to be modeled.

SSH in Modeler Plugins

In order to make a modeler plugin utilize SSH as it's polling method we need to use a different type of plugin, the CommandPlugin plugin. The structure of this type of plugin is very similar to the SnmpPlugin, however there is one critical variable that must be implemented.

Working With Zenoss Events

Fri, 25 Nov 2016 00:00:00 +0000

Zenoss Events are among the most powerful and useful monitoring features that Zenoss Core provides. Many different types of events are automatically generated by Zenoss when important scenarios occur for the devices being monitored.

Zenoss Event Architecture

When an event arrives at Zenoss, it is parsed, associated with an event classification and then typically (but not always), it is inserted into the status table of the events database. Events can then be viewed by users using the Event Console of the Zenoss Graphical User Interface (GUI).

The events system has the concept of active status events and historical events (two different database tables in the MySQL events database).

Understanding Event Classes

Zenoss event classes are a simple organizational structure for the different types of events that Zenoss generates and receives. This organization is useful for driving alerting and reporting. You can, for example, create an alerting rule that sends you an email or pages you when the availability of a Web site or page is affected by filtering on the /Status/Web event class^[2].

The Zenoss JSON API

Thu, 24 Nov 2016 00:00:00 +0000

The Zenoss JSON API allows us to obtain very important information of what is going on in Zenoss, such as device information and events. This API can be queried using cURL or with some wrappers provided by Zenoss, available in languages such as Bashscript, Python, and Java.

The API documentation can be downloaded here. In the documentation you can see the available endpoints and methods that can be used to obtain the data we need.

In this post I will cover how the Zenoss back-end JSON API works, and how the Zenoss's front-end (made using ExtJS) interacts with it.

~> Even though Zenoss calls its API a JSON API (Simply because it returns data in JSON format), the API is not JSON API specification compliant.

Querying the API

To see the API in action, we can begin by making a simple query using cURL. Let's say we want to obtain all the available information of a specific device:

curl -u "admin:zenoss" -X POST -H "Content-Type: application/json" -d '{"action": "DeviceRouter", "method": "getInfo", "data": [{"uid": "/zport/dmd/Devices/Server/SSH/Linux/NovaHost/devices/$DEVICE_IP"}], "tid": 1}' http://$ZENOSS_HOST:8080/zport/dmd/device_router

The above request will return the information of the device in JSON format, assuming that we replace $ZENOSS_HOST with the IP address of our Zenoss server, and $DEVICE_IP with the IP address of the device we want to query.

This is all fine if all we need is querying the API and nothing else. But what if we really want to know how the API back-end works? How the endpoints are created? What are routers?

This is what we are gonna learn next.

Using RRDTool in Zenoss

Tue, 22 Nov 2016 00:00:00 +0000

rrdtool is an awesome high performance data logging and graphing tool for time series data. Zenoss Core uses RRDTool to collect, monitor, and graph peformance data for devices.

However Zenoss Core comes with built-in helper wrappers around RRDTool that makes using it within Zenoss much easier. These source code files can be found in $ZENHOME/Products/ZenRRD/.

RRDUtil

Located in $ZENHOME/Products/ZenRRD/RRDUtil.py, this Python module contains many wrapper methods around the rrdtool library. These methods can help us write to, create new, and read .rrd files using Python.

We can easily import this module into our code, using the following import statement:

from Products.ZenRRD.RRDUtil import RRDUtil

Creating Zenoss ZenPack Daemons

Mon, 07 Nov 2016 00:00:00 +0000

ZenPacks are powerful custom add-ons that can help us extend Zenoss's functionality. In this post I will go over on how to create a ZenPack that adds a custom daemon to the existing Zenoss daemons and runs on a configured cycle time to perform custom tasks. We will achieve this by creating the ZenPack using zenpacklib, but it is also possible to create it from the Zenoss user interface.

About zenpacklib

zenpacklib is a Python library developed by the Zenoss team to facilitate the process of creating ZenPacks, specially ZenPacks that deal with modeling and monitoring devices and components. Most of the newer ZenPacks that are being released are now being built with zenpacklib.

We can obtain zenpacklib by running the following commands:

wget http://zenpacklib.zenoss.com/zenpacklib.py
chmod 755 zenpacklib.py

This will download the zenpacklib Python library and give it executable permissions.

Creating the ZenPack

Using the zenpacklib file we just downloaded, we proceed to create a new fresh ZenPack:

./zenpacklib.py create ZenPacks.<your_namespace>.<zenpack_name>

This will create the ZenPack base directory with the necessary base files.

Now we go into this directory to begin.

cd ZenPacks.<your_namespace>.<zenpack_name>

Creating the ZenPack Daemon

Our custom daemon declaration will be located in a directory named daemons, inside our ZenPack directory:

mkdir ZenPacks.<your_namespace>.<zenpack_name>/<your_namespace>/<zenpack_name>/daemons

Here we will create a new Bashscript file with the name of our daemon. In this case we will name it mydaemon:

#! /usr/bin/env bash

DAEMON_NAME="mydaemon"

. $ZENHOME/bin/zenfunctions

MYPATH=`python -c "import os.path; print os.path.realpath('$0')"` THISDIR=`dirname $MYPATH` PRGHOME=`dirname $THISDIR` PRGNAME=$DAEMON_NAME.py CFGFILE=$CFGDIR/$DAEMON_NAME.conf

generic "$@"

Once the ZenPack is installed, files under this daemons directory will become executable (chmod 0755), a symlink to the file will be created in $ZENHOME/bin, and a configuration file will be generated in $ZENHOME/etc/<daemon_name>.conf

NOTE: If you created your ZenPack using the Zenoss user interface, the daemons directory will also be automatically created, and will contain an example daemon file named zenexample with code similar to the one above. In this case you should simply replace the necessary values.

Zenoss Monitoring Template Data Points

Thu, 03 Nov 2016 00:00:00 +0000

Content from this post is mostly obtained from Zenoss Core Administration guide.

In Zenoss Core monitoring templates, data sources can return data for one or more performance metrics. Each metric retrieved by a data source is represented by a data point.

When creating a data point, there are some important fields that we can define for our data point:

Name: Displays the name you entered in the Add a New DataPoint dialog.
RRD Type: Specify the RRD data source type to use for storing data for this data point. (Zenoss Core uses RRDTool to store performance data.) Available options are:
- COUNTER: Saves the rate of change of the value over a step period. This assumes that the value is always increasing (the difference between the current and the previous value is greater than 0). Traffic counters on a router are an ideal candidate for using COUNTER.
- GAUGE: Does not save the rate of change, but saves the actual value. There are no divisions or calculations. To see memory consumption in a server, for example, you might want to select this value.
- DERIVE: Same as COUNTER, but additionally allows negative values. If you want to see the rate of change in free disk space on your server, for example, then you might want to select this value.
- ABSOLUTE: Saves the rate of change, but assumes that the previous value is set to 0. The difference between the current and the previous value is always equal to the current value. Thus, ABSOLUTE stores the current value, divided by the step interval.
Create Command: Enter an RRD expression used to create the database for this data point. If you do not enter a value, then the system uses a default applicable to most situations. For details about the rrdcreate command, go here.
RRD Minimum: Enter a value. Any value received that is less than this number is ignored.
RRD Maximum: Enter a value. Any value received that is greater than this number is ignored.

Data Point Aliases

Performance reports pull information from various data points that represent a metric. The report itself knows which data points it requires, and which modifications are needed, if any, to put the data in its proper units and format.

The addition of a data point requires changing the report.

To allow for more flexibility in changes, some reports use data point aliases. Data point aliases group data points so they can be more easily used for reporting. In addition, if the data points return data in different units, then the plugin can normalize that data into a common unit.

Changing Zenoss Dashboard Portlets Using CLI

Wed, 26 Oct 2016 00:00:00 +0000

The Zenoss ZenPacks.zenoss.Dashboard ZenPack adds a very nice and new dashboard to our Zenoss Core 4 deployment, replacing the old default dashboard that comes with the installation.

Creating Python Packages

Thu, 13 Oct 2016 00:00:00 +0000

For a project involving Zenoss Core 4 and a HaaS solution maintained by another team, we needed a module to interact with the Zenoss JSON API to get the list of events for specific devices. Browsing around I found this python-zenoss module to work with the Zenoss JSON API. However I was experiencing some issues when installing it, so I decided to create my own Python package to provide a different way to interact with the Zenoss JSON API according to our specific needs.

The package would be installed locally using pip, and all its functionality should be easily accessible using from and import commands in Python.

To start working in our package we will create a base directory. The directory's name will resemble our package's name. For naming, we should following these guidelines:

All lowercase
Underscore-separated or no word separators at all (don’t use hyphens)

The package contents are the following:

my_api/
├── __init__.py
├── api.py
└── setup.py