OpenGroupware Coils Optional Dependencies

Installing OpenGroupware Coils via easy_install or pip will automatically install a variety of required dependencies.  There are however serveral optional dependencies.  The server will run without the availability of these modules but some features will be automatically disabled.  Installing the following modules will extend the capabilities of your OpenGroupware Coils instance.

• informixdb - Allows an Informix RDBMS to be used as an external data source.  This means you can define an Informix database via OIESQLDataSources and utilize it for SQL related workflow actions such as sqlInsertAction, sqlSelectAction, etc... 
• z3c.rml / zope.interface - Installing these modules will enable the conversions of XML to PDF.  Using XSLT transforms you can turn your data into RML, and then using the rmlToPDF workflow action convert the data into sophisticated PDF documents.
• pysmbc - Enables the OIE workflow engine to retrieve and upload files from or to a CIFS / SMB volume. 
• PIL - Enables the OSSF module for retrieving thumbnails of images stored in OpenGroupware projects or attachments. 
• paramiko - Presence of this module activates the SSH related OIE workflow actions such as sshExecuteCmmand. 
• python-ldap - The availability of the LDAP module enables both support for LDAP authentication (such as when using Active Directory, OpenLDAP, etc... as your user database) and support for querying an LDAP DSA from a workflow process using OIE's ldapSearchAction.

Once your OpenGroupware Coils instance is up and running you should evaluate which additional, optional, dependencies would be useful in your environment.

0.1.49rc7 & CardDAV-Sync

With the release of 0.1.49rc7 comes a collection of bugfixes and enhancements - most notably is the early arrival of service discovery support for the CardDAV-Sync Android application.  CardDAV-Sync, as the name implies, adds CardDAV support to an Android device. 

OpenGroupware Coils 0.1.49rc7 has been uploaded to PyPI and SourceForge.

In order to provide easy configuration CardDAV-Sync uses the well_known standard - so you can avoid entering tedious WebDAV paths to collections of contacts (aka address books).  The new well_known support in 01.49rc7 allows CardDAV-Sync to discover the location of the address books and present them clearly to the user.  So configuring a CardDAV-Sync client requires nothing more than the host name, user name, password, and selecting the address book you want on your device.  Of course you can create multiple accounts in order to have multiple address books.

Thanks to service discovery CardDAV-Sync configuration is simple.

This release also changes the WebDAV hierarchy for presenting contact collections to clients - this is necessitated by the new support for user-defined addressbooks.  Now all contact collections are available under "/dav/Contacts" and the previous contents of "/dav/Contacts" (all the server's contacts) are moved to "/dav/Contacts/All Contacts". 

For convenience the "All Contacts" addressbook is hidden from the CardDAV-Sync client (based on user agent detection) - as all contacts can be simply too large a collection for easy use on a small and resource constrained device such as a phone.

The address books "Favorites" and "Users" are fixed and cannot be renamed or deleted by the user - these contain favorite contacts and all the server's users, respectively.  The "Users" address book is read-only.  Creating a contact in the favorite address book will automatically make that contact a favorite.
New address books can be created via WebDAV by simply creating a new collection [folder] in the "/dav/Contacts" collection [folder].

The contents of /dav/Contacts viewed in GNOME's Nautilus file browser.  New address books can be created simply by creating a new folder.

Android 4.x users can use CardDAV-Sync alone, but users on previous versions of Android must all install the ContactEditor application as earlier Android versions did not permit the editing of contacts on third-party servers. 
Even user's of 4.x devices may want to look at DMFS' excellent CardDAV-Sync Pro and ContactEditor-Pro applications as they extend the capabilities of contact management on Android.  If you are using a device linked to a proprietary marketplace these applications should be available there, otherwise they can be acquired from the AndroidPIT.
Technical information regarding support forthe CardDAV-Sync client can be found at that client's wiki page of the OpenGroupware Coil's development wiki.

GRPUG Presentation on OIE

Last night I presented at the Grand Rapids Python Users Group about the OpenGroupware Coils project and specifically the workflow component: OpenGroupware Integration Engine.

Click to download presentation (PDF).

This presentation convers the basic terminology and concepts behind the OpenGroupware Integration Engine - how to think about process modelling with OIE.  For full details on how to use OIE, as always, see the WMOGAG document.

Note: the OIE section of the presentation is featured above. The full presentation, as given, with a few more slides and some general Python bits is also available here.

New Edition of WMOGAG Coils Available

A new edition of WMOGAG Coils [Whitemice Consulting's OpenGroupware Administrator's Guide] has been uploaded to SourceForge; download here.  This edition documents the new features available in recent production and RC releases include using the workflow scheduler, the new translateAction, and configuration of the services for SSH access to other hosts.
As always please report any comments, recommendations, or even typographic errors to either the Coil's SourceForge project or the Coil's maillist.

The Workflow Scheduler

With the 0.1.49 release [and currently available in the 0.1.49rc releases] OpenGroupware Coils exposes the workflow scheduler via WebDAV at the path "/dav/Workflow/Schedule".  This collection [folder] allows enumeration of schedule entries by WebDAV clients or via REST, as well as the creation and deletion of schedule entries. 

Each entry is represented by a JSON serialized resource.  These resources can be retrieved via GET, created via PUT, or removed via DELETE. For REST clients the resource "/var/Workflow/Schedule/.contents" will return a JSON encoded list of all the folder entries.  Via either RPC mechanism clients only see the schedule entries available to their own security context.

$ cat send-request.json
{"routeId":1158480,"priority":300,"repeat":1,"type":"interval","minutes":2}
$ curl -vvvv -u adam:******* -X PUT --upload-file send-request.json http://127.0.0.1:8080/dav/Workflow/Schedule/request
...
< Location: /dav/Workflow/Schedule/0f6bd804a4f14cf6a2807dad625e2bb1.json
< X-OpenGroupware-ScheduleEntryUUID: 0f6bd804a4f14cf6a2807dad625e2bb1 
...
Create a scheduler entry where an instance of the route with the objectId 1158480 will be created two minutes from the current time having a priority of 300.  The schedule will only occur once.

The required keys in the JSON data are: "routeId" and "type".  The keys "priority", "contextId", "xattrDict", "attachmentUUID", and "repeat" are optional.  If unspecified the priority for scheduled entries will be 200, they will repeat without limit according to their schedule, and have the context of the currently authenticated user.  A user may only create schedule entries with contexts that are available to them. Additional keys are required or optional based upon the value of "type" which describes the kind of scheduling entry to create: "simple", "interval", or "cron".

If the workflow requires an input message an attachment may be created via the AttachFS protocol and the UUID of that attachment then specified in the scheduling entry; at run time the content of the attachment will be read into the input message of the new process.  The attachment's UUID is sent in the  "attachmentUUID" attribute of the scheduling request.

In the future it will also be possible to specify a project document as an input message.  Project documents have the advantages over attachments that they have a security descriptor and can be modified in-place.

Using the "xattrDict" attribute a dictionary of key and values can be declared that will be applied to a new process as its XATTR values.

An "interval" schedule type must specify at least one of the following values: "weeks", "days", "hours", "minutes", "seconds".  The type of any of these values must be an integer.  Multiple values may be specified and their accumulative value will specify the frequency the process will be run.

A "simple" schedule requires on a "date" value; this specifies a specific time when the process will be created.  "simple" schedule entries do not repeat. 

The "cron" type schedule entry requires at least one of the following values: "year", "month", "day", "weekday", "hour", or "minute".  All these are crontab style pattern matching strings; both division and comma separated lists are supported.  Any of these values if not specified default to the string "*".

All date/time values must be in UTC and expressed in either "%Y%m%dT%H%M%S" or "%Y-%m-%d %H:%M:%S" format. In either format the seconds value is optional, and if provided is ignored.  Remember that the OIE scheduler is a best-effort scheduler - it will attempt to create the process within a minute of the time indicated by the schedule, but delay is possible.  Also the scheduler only creates the process and queues it for execution.  It is up to the workflow manager component when the process will actually be executed/performed.  A variety of factors can influence the timing of process's execution, including singletons, system load, and administrative holds.

A schedule entry retrieved from the server will have all possible fields filled in as well as having two additional read-only fields: "iterationsPerformed" and "iterationsRemaining".  "iterationsPerformed" records how many times the schedule entry has been executed and "iterationsRemaining" indicates how many addition additional iterations remain.  A value of -1 in "iterationsRemaining" means the schedule entry will continue indefinitely; this field will only have a real value if "repeat" was specified when the scheduler entry was created.  If the entry has a repeat limit the "iterationsRemaining" serves as a count down to the expiration of the entry, once the remaining iteration count reaches zero the schedule entry will be automatically deleted.

{"dayOfWeek": "3,5", "week": "*", "nextIteration": "2012-11-11T23:00:00", "month": "*", "second": "0", "iterationsRemaining": -1, "year": "*", "day": "*", "minute": "0", "attachmentUUID": null, "xattrDict": {"start": "$__MONTHSTART__;", "end": "$__TODAY__;"}, "contextId": 15211340, "UUID": "{c658993a-dcf1-410b-8003-5dc134feb564}", "hour": "23", "routeId": 70470079, "priority": 200, "iterationsPerformed": 7, "type": "cron"}
A complete schedule entry as retrieved from the server.  Each time this entry creates a process that process will have a "start" XATTR whose value represents the date for first of the current month and an "end" XATTR that represents the current date. 

For additional flexibility the OIE built-in labels are supported in the value of process XATTRs (specified in the schedule entry's "xattrDict" value).  The values will be substituted each time a new process is created from the schedule entry.  Using built-in labels is especially useful when creating schedules for workflow processes that require a date range.

Process XATTR's and label substitution is documented in the Coils edition of WMOGAG.

Snurtle: get-performance

The latest version of OpenGroupware Coils + Snurtle provides a new command "get-performance". get-performance will retrieve the performance counters recorded by the coils.adminstrator component concerning Logic layer operations.

Output is a single line for each Logic operation that includes its frequency, average duration, minimum and maximum duration, and the total system time that has been consumed performing that operation..

Aside: Snurtle's command completion only requires you to type enough of a command to make the command unique, so "get-perf" is plenty.

Snurtle has also recently added the commands lock, unlock, list-locks, and create-contact.  The built-in help covers the syntax for these as well as get-performance.

Makefile Deployment

In the latest commits to OpenGrouwpare we've added "Makefile deployment".   This allows for simple and rapid creation of a development instance.

So long as you have PostgreSQL and RabbitMQ available you can just

1. Check out the code
2. Tweak the first few lines of the Makefile to tell it your PostgreSQL database name, user name, etc...
3. If you already have a database then touch 0_database_complete.txt to indicate that the database has already been initialized.
4. To fire-up a working test instance just run "make run-master"
5. Point your WebDAV client at 127.0.0.1:8080/dav

That's it! The make file will create a Python virtualized container to handle dependencies, retrieve the required dependencies, install a bare minimum configuration, create a document root, and start the service.  You will have the administrative account with the password you specified in the first few lines of the make file.

You can run "make install-dependencies" if you want to update the Coils dependencies check  - just delete the 0_dependencies_installed.txt file which indicates that dependencies have already been isntalled.

Aside: You may also want to add some of the optional Python modules to your virtual environment such as PIL, informixdb, z3c.rml, procname, or YaJL.  These are not required by OpenGroupware Coils and are not automatically installed.

Simple steps to provision PostgreSQL & RabbitMQ

sudo sudo -u postgres createuser --password --no-superuser --no-created --no-createrole OGoDev
sudo sudo -u postgres createdb -E UTF-8 -O OGoDev OGoDev0
sudo /usr/sbin/rabbitmqctl add_user OGoDev {AMQPASSWORD}
sudo /usr/sbin/rabbitmqctl add_vhost OGoDev0
sudo /usr/sbin/rabbitmqctl set_permissions -p OGoDev0 OGoDev ".*" ".*" ".*"
Text1: Provision PostgreSQL and RabbitMQ

This will give you a PG and AMQ user "OGoDev" with the password specified, a database named "OGoDev0", and an AMQ virtual host named "OGoDev0".  These values are set in the top of the Makefile.