In the two previous posts we commented the main characteristics of the EMS Bus and described Calduino, the hardware and software used to connect and exchange information with a Buderus boiler though this bus. In this post you will see how to create a weeWX extension to import the information gathered and how to control the boiler configurations from an HTTP web page.
Creating a WeeWX Extension
WeeWX has a data/event driven architecture. It uses different drivers to communicate with the connected weather stations (hardware). When a sensor spits some data out, weeWX does something. This action can be either generate an HTML report or perform some calculations with the data gathered. These actions are called services.
The drivers are Python code that reads from serial port/USB/network interface the bytes sent by the hardware and decodes them, putting the resulting data into LOOP packets and archive RECORDs. In my case, I use the WMR89 driver to recieve the meteorological conditions. A weeWX instance can only have one driver, so in order to incorporate a new source of information, one or more services should be incorporated.
WeeWX services are bound to events and there can be more than one service bounded with the same event. There are different types of services, listed in the following table:
|prep_services||Performs any actions before the main loop is run.|
|data_services||Augments data, before it is processed.|
|process_services||Processes, filters, and massages the data.|
|archive_services||Records the data in a database.|
|restful_services||Uploads processed data to an external RESTful
|report_services||Runs any reports.|
In general terms we want to augment the data included in the archive RECORD and save it in a database. To so do, two data services will be needed:
- First of all, a data service (called CalduinoMonitor) will include Calduino variables in every archive RECORD generated. In my case, the archives are generated in a five minutes pace (archive_interval parameter in weewx.conf). This can be achieved in two ways. The easiest one would be to send a Get All Monitors command to calduino (operation 29) and parse the XML results. However this get operation takes on average 18 seconds to be performed, blocking weeWX engine during this time. To avoid this I have created a Linux service that downloads the output of this get command every five minutes, before being imported by weeWX. By doing this, the data service will only have to access the last XML file and import the values.
- At the end, an archive service (called CalduinoArchive) will store the data in a new database that will only contain the information gathered by Calduino. I have decided to create a new database and not to extend the default database following WeeWX recommendation: extensions should not modify the database schemas.
As it has been commented before, a new database will be used to save Calduino data. It will just contain a number of the parameters included in the Get All Monitors command. Not all the parameters contained in this command will be saved. Some of them are just current configurations/status of the boiler. It may be of interest to show them in the HTML report in real time but it makes no sense to save them for the future.
The following table summarizes Calduino’s database schema. Note that the field names coincides with the used in the XML tags:
|dateTime||Integer||Not null, primary key (weeWX default field)|
|usUnits||Integer||Not null (weeWX default field)|
|interval||Integer||Not null (weeWX default field)|
The last service being run in every weeWX iteration is weewx.engine.StdReport. This is the standard service for creating reports. Each report has a skin associated with it, which contains the templates, any auxiliary files such as background GIFs or CSS style sheets, and a skin configuration file, named skin.conf. In order to see the data gathered and manage remotely the boiler, a new skin, named calduino and packed in this extension, has been developed.
To create their output skins rely on one or more generators, which are those that do the actual work, such as creating HTML files or plot images. The Calduino Skin included in this extension is just a proof of concept and uses only two generators:
- The CheetahGenerator will generate the HTML web page files from templates, using the Cheetah template engine. A template is a text file, ended with the .tmpl extension by convention, that is processed by a template engine to create a new file. The Cheetah template engine finds tags by scanning a search list, a Python list of objects. To include Calduino tags in the search list, I have created a search list extension called CalduinoSearchList. This will make some of the data imported from Calduino available in the report but not saved in the database, such as Calduino’s uptime, number of operations failed, etc.
Additionally, you may also use other generators such as the GaugeGenerator to show, in a friendly way, the information gathered by Calduino. I have developed, based on the work of Nick Dajda’s Gauge Generator, a type that can include up to three observations from different data sources. More information can be found in this project.
Finally, you will surely want to use the ImageGenerator to create graphics regarding the boiler status. It is up to you decide which generators you want to run and what information you want them to provide.
Installing Calduino Extension
To install the extension using the
wee_extension utility follow this steps:
- First of all download the last calduino extension from the releases tab on the project GitHub site and save the downloaded file to a directory accessible from the machine running weeWX. In these instructions the /tmp directory will be used.
Shell1wget -P /tmp https://github.com/danimaciasperea/weewx-calduino/archive/v1.0.tar.gz
- Run the installer with a user with privileges. This command assumes that wee_extension is included in user’s path, otherwise the path to wee_extension will be required.
Shell1sudo wee_extension --install=/tmp/weewx-calduino-1.0.tar.gz
This will result in output similar to the following:
Shell12345Request to install '/tmp/weewx-calduino-1.0.tar.gz'Extracting from tar archive /tmp/weewx-calduino-1.0.tar.gzSaving installer file to /usr/share/weewx/bin/user/installer/calduinoSaved configuration dictionary. Backup copy at /etc/weewx/weewx.conf.2016052723124410Finished installing extension '/tmp/weewx-calduino-1.0.tar.gz'
- Adapt the configuration files to your installation. Update, if required, the weewx.conf to change the location of the XML files to import in the [Calduino] section. Default configuration presumes an
sqlitedatabase. Change the
database_typeattribute in [Databases] section to mysql if needed.
- The tarball file contains an example of service to obtain from the boiler periodically its status in XML format. It is located in
/tmp/pollCalduinofolder. It is highly tailored to my needs, so most likely you will have to update it. More information about this issue in this section.
- Restart weeWX:
Shell1sudo service weewx restart
In the first run WeeWX will create a new database named calduino.sdb with the previously commented schema. As stated above, the Calduino Skin and templates included in this extension is just a proof of concept with very few options. Edit the template files to include graphics and/or gauges of your boiler as desired.
calduinoWrapper.php.tmpl performs the communication with the Calduino device using
The next image is an example of Calduino Extension running on my weeWX.
Poll Calduino Service
Included in the extension, in
/tmp/pollCalduino folder you will find a bash script called
pollCalduino.sh. It will be in charge of polling Calduino device connected to the EMS Bus and updating the XML file that will be imported by weeWX’s extension. To do so, it will launch the php operation
getGeneralMonitor every minute ended in 3 or 8 (as explained in this post). To work properly it should include the IP address of Calduino in
When a communication problem is detected (i.e. the operation fails), an IFTTT notification is sent to my telephone. The steps done to configure your IFTTT account were explained in this entry. You should just create a service named
calduino_error and include your IFTTT key in the script. Moreover, I use a 433 MHZ transmitter module installed in the Raspberry to remotely turn off and on the calduino device, as it is connected to a radio frequency controlled socket. More information about this can be found here. If you are not going to use this functions just comment these lines.
- It happens infrequently, but sometimes the pollCalduino operation fails due to collisions in the EMS Bus. The operation is retried up to 5 times to avoid this issue. In case the problem is related to the WiFi network, a restart (switching off and on the RF socket) is performed as it will probably solve the problem.
- It should be possible to pass the logic of the Poll Calduino Service to weeWX. Calling directly calduino from python is not a good idea, since this code sits right in the main loop of the weeWX engine. Considering that it takes up to 18 seconds to obtain a general monitor file, a different approach should be used. For instance, this code could be included in a separate thread and feed the results to Calduino Monitor through a queue. I left this improvement to the future.