Friday, July 27, 2012

diazotheme.bootstrap 0.2 released

I have been using diazotheme.bootstrap for a number of community sites for quite a while now. It was originally developed around November last year with Bootstrap 1, and have gone through some updates and enhancements to Bootstrap 2 , but it only remains in its github repository, but never actually released properly.

Anyway~ ... after long due, diazotheme.bootstrap 0.2 is now released in PyPi: http://pypi.python.org/pypi/diazotheme.bootstrap/0.2

  

Btw, should I fork this into collective github?

Thursday, July 26, 2012

Virtual hosting on Grok (or Bluebream) - without a reverse proxy

When I was poking at creating the Grok template for OpenShift, I needed to add virtual hosting for the test application.

However, OpenShift's DIY cartridge does not allow me to modify the reverse proxy settings as they are automatically generated by the cartridge. Naturally, I had to find another way.

Grok virtual hosting with WSGIRewrite

WSGIRewrite is a WSGI middleware which provide a facility for mod_rewrite compatible rewrite rules for WSGI applications. Using this middleware, we can rewrite the URL to force virtual hosting on the application server.

Lets go straight to the meat, here is the howto.

Howto

The first step is of course, to add WSGIRewrite into your buildout. Edit setup.py and add 'WSGIRewrite' into the install_requires list, and re-run buildout.

Afterward, edit your WSGI INI file, (eg: etc/deploy.ini.in), and find a section called pipeline:main, and add rewrite as the first item in the pipeline. The section should appear like this now:

[pipeline:main]
pipeline = rewrite accesslogging evalexception fanstatic grok

At the bottom of the file, add these lines. Replace examplesite with an identifier for the entry, replace app with your application ID, and replace www.example.com with the virtual hosting domain:

[filter:rewrite]
use = egg:WSGIRewrite
rulesets = examplesite-http examplesite-https

[wsgirewrite:examplesite-http]
cond1 = %{HTTP_HOST} ^www.example.com$
cond2 = %{HTTPS} off
rule1 = ^/(.*) app/++vh++http:www.example.com/++/$1

[wsgirewrite:examplesite-https]
cond1 = %{HTTP_HOST} ^www.example.com$
cond2 = %{HTTPS} on
rule1 = ^/(.*) app/++vh++https:www.example.com/++/$1
 
Rerun buildout to regenerate your INI files, and start your paster/wsgi server. Access the server using the configured domain, and it should work.

Wednesday, July 25, 2012

Grok framework quickstart template for OpenShift using the DIY cartridge

Another OpenShift goodness for Zope developers. A quickstart template for the Grok framework! Grok is now deployable easily on openshift.redhat.com!

Check it out here: https://github.com/kagesenshi/grok-openshift-quickstart

Grok on OpenShift

Unlike the Plone template which is pretty straightforward, the Grok template requires some additional steps to initialize the project. This is mainly due to Grok projects are packaged in python eggs by default, so the package name is quite important. A preconfigured package name would surely cause name clashes.

To initialize the project, just run initialize.py with your project name as the parameter. The script will automatically install grokproject, create your project, move the files around so that it works nicely with OpenShift, and add the necessary files into the git repo.

Features
  • Create a grok project using your own custom project name
  • Deploy your application using the environment variables of OpenShift
  • Build process, caching and storage is implemented the in a very similar way with the Plone template I created earlier.

Tuesday, July 24, 2012

Plone on OpenShift quickstart using the DIY cartridge


TLDR; Plone is now deployable easily on openshift.redhat.com . Check out the repository here: https://github.com/kagesenshi/plone-openshift-quickstart.

Plone on openshift.redhat.com

Yesterday I posted about  deploying Plone using a cartridge in your own private OpenShift Origin VM.

Afterwards that day, I had some chat in the #openshift IRC channel and was informed that it is OK to run buildout in openshift.redhat.com, and last night, I converted the scripts and config from the cartridge so that it fits in the workflow of the DIY cartridge.

Features/Notes:
  • Deploys Plone 4.2 with Dexterity content types easily and quickly on openshift.redhat.com
  • Downloads are cached in ${OPENSHIFT_TMP_DIR}/${OPENDIR_APP_UUID}-cache. However, as this is not shared, it means all new apps will have to redownload the python eggs. While it might be possible to use a shared download cache for this  (OPENSHIFT_TMP_DIR is apparently /tmp), I am not sure whether the SELinux settings in the server allows it or not.
  • Virtualenv and generated eggs are stored in OPENSHIFT_DATA_DIR. Technically though, they are not exactly data. However, as we need them to be persistent, and the DIY cartridge does not provide a folder to store persistent, non-data, automatically generated files, they had to be stored there.
  • The quickstart template is quite generic for any buildout-based deployment. So, technically, you can add support for Grok, BlueBream and any framework that can be deployed using buildout using this template as its base. I'm planning to create a quickstart template for deploying Grok either tonight or later this week.  Will post about it.

Monday, July 23, 2012

Plone cartridge for deployment on OpenShift

I was poking around OpenShift on last weekend to see what it offers, and also to see whether I can deploy Plone on it.

TLDR; Plone deploys well on my OpenShift VM. I created a cartridge for it here: https://github.com/kagesenshi/cartridge-plone . I can now easily add new Plone deployments for my development server with less work.

About OpenShift

OpenShift is a Platform as a Service (PaaS) software from Red Hat. It can be summarized as a FOSS Heroku-like platform for deploying web applications easily.

Support for applications comes in 'cartridges', which basically a set of files and scripts to prepare a self-contained environment for your application deployment. OpenShift includes several official cartridges to deploy popular applications and frameworks such as Drupal, RoR, Django, WSGI, etc. There is also a cartridge called 'DIY' which allows you to import any http daemon and serve it through OpenShift - thus providing a basic way in integrating non-officially supported frameworks.

Plone on OpenShift

Due to Plone is rather huge and also the uncertainty on what the kind of problems I might face trying to deploy Plone in openshift.redhat.com, I created my own private OpenShift server following this guide https://openshift.redhat.com/community/wiki/build-your-own-paas-from-the-openshift-origin-livecd-using-liveinst. At least this way, I have easy access to the server to inspect codes and error logs when things does not work.

Afterwards I went digging and experimenting on existing cartridges to understand how things work (Official doc: https://openshift.redhat.com/community/wiki/introduction-to-cartridge-building) and derived the first plone cartridge using bits from python-2.6 and diy-0.1 cartridges.

After hours of experimenting and hacking around, the plone cartridge is fully functional to deploy basic Plone site. With the cartridge installed, I can now easily add new single instance Plone sites with just this command:

rhc app create -t plone-4.2 -a myplonesite 

The cartridge

I have uploaded the Plone cartridge code here: https://github.com/kagesenshi/cartridge-plone . Follow the readme on how to build the RPM and install it into your OpenShift server. There is one annoyance though; the cartridge list is apparently hardcoded into a ruby file, making it somewhat annoying to automate the final step of the installation (would be nice if theres a config file for this, and ruby is like perl to me - difficult to read when the dev uses the language fully). So remember to follow the guide on adding your cartridge into the code and reload the cache.

Do check it out ^-^

The good and the not-so-nice
  • The cartridge works pretty well in deploying new basic plone site
  • Building new cartridge is easy and straightforward
  • Many Plone deployment in one server and they are not fighting for port/address - WIN!
  • The automatic DNS configuration is very neat - especially the 'add-alias' command. 
  • The Plone cartridge have been configured to share download-cache, but I'm not sure on the security implication.
  • Looking at how things are deployed, it made me wonder on security of a multi-user, non-private openshift server. How does one know that an app won't mess with the filesystem. Or is this using SELinux?, If yes, what are the stuff allowed,and what are not?.
  • Due to some security concern and possible umask/permission problem, I did not configure eggs-directory to be shared. This affects the speed of deployment, and you lose your eggs directory when you destroy an app. New apps will have to rebuild its eggs cache.
  • There is a 'snapshot' command which allows you to download your data, however I'm not sure how reliable it is for large data.
  • Instance is stopped and buildout is re-run for every git push and also when doing snapshot. This can cause downtime for sites. For snapshots, it would be nice if theres a way to rsync without stopping the instance.
  • The 'tail' command (tails the log) is cool. 
  • The 'tidy' command and hook looks useful, perhaps to do zeopack and log shortening. The cartridge yet to have any feature related to this. 
  • '/health'  URL is reserved by OpenShift to check whether the site is up, no idea whether this can be changed to something thats harder to clash (eg: openshift-health or something)
  • I'm not sure whether it is possible, or allowed to run buildout using the DIY cartridge on openshift.redhat.com. If it can be done, then this cartridge can be simplified to hooks for the DIY cartridge.
  • Perhaps I should create a Plone+ZEO cluster cartridges later, so that this become more scalable.
  • It would also be super cool if this cartridge could become part of the standard offering in openshift.redhat.com. In the meantime, I shall use it in Inigo's internal test server (or perhaps offer this as proper host later once it easily installable through EPEL or something)

UPDATE: You can now deploy Plone on OpenShift using the DIY cartridge. Check out this post: http://blog.kagesenshi.org/2012/07/plone-on-openshift-quickstart-using-diy.html