Pylons

restler - RESTful controller for Pylons: For those looking for a RESTful setup for Pylons that is similar to Django-Piston.

Ian Bicking recently released a rather interesting package called toppcloud that aims to tackle what I see as a growing need for those of us deploying Python webapps. I was actually interested in easing my own deployment woes before I saw Ian’s ... [More] announcement about his package, and was halfway through a rather hefty amount of research on automating server deployments with tools like Chef and Puppet, but toppcloud is a bit different. It not only tackles provisioning a new system on the fly from the ‘cloud’ (using libcloud), but it also handles easy Python (and now PHP) web application deployments. With such a tantalizing set of goals, I couldn’t really resist getting my feet wet. Boy am I glad I did. PylonsHQ is now running with toppcloud, and I won’t be surprised when more people get it running for them. When many shared hosting providers are $5-10 a month, its rather nice to pay $10/mth for an automatically configured VPS, with one-command deployments. Though of course, unless you go to quite a bit of work yourself, most shared hosting providers don’t have one-command deployments for you. Before I continue on to describe how I setup Kai - the source code behind PylonsHQ - I should provide a few caveats about using toppcloud: toppcloud is alpha software, there’s no releases of it yet, you will be checking it out from source code currently, only Rackspace Cloud is known to be working, though since it uses libcloud, in theory, any cloud providers that it supports should be usable toppcloud is changing rapidly, be ready to keep up on the commit log to see whats changing there are no unit tests, most likely because its very tedious to make the rather significant amount of mock objects required to test the various local/remote commands and the fact that its changing so fast the tests would probably be obsolete in a week toppcloud philosophy I’m half guessing here, based on talking with Ian and reading the docs myself, but the philosophy of toppcloud is around providing a common deployment platform, ala Google App Engine, except of course with Postgres or other ‘services’ that you can request to use. At the moment the only services that toppcloud comes with is CouchDB, Files (to store/serve files for your app on the filesystem), and Postgres w/postgis extensions. For those diving into the source code, it shouldn’t be too hard to see how to create additional services and I hope to see more get added as people get more interested. Therefore, toppcloud is not expected to be everything to everyone, it is expected to be at least an 80+% solution to deploying web apps in a Google App Engine style ease-of-deployment process. toppcloud itself then ensures that depending on what service you asked for, its setup and ready for use on the server when your app is deployed. If that sounds like something you’re dying to try out, you’re in luck! Setting up a Pylons App I’m not going to mention how to check out the toppcloud source, except to mention the directions are in the /docs/index.txt file in the toppcloud source (which should be read in its entirety!). Once you have toppcloud installed on your computer, getting from zero to running website on VPS is remarkably quick: $ toppcloud create-node --image-id=14362 baseimage Wait until email arrived indicating that server is up and ready $ toppcloud setup-node baseimage Server is now all setup to run web apps! $ toppcloud init myapp Create an app to deploy to our server $ source myapp/bin/activate Activate the virtualenv used for this app $ pip install Pylons $ cd myapp/src $ paster create -t pylons awesomeapp Create our Pylons app, or check out an existing one from your VCS here $ cd ../.. $ ln -s myapp/src/awesomeapp/awesomeapp/public/ myapp/static toppcloud will make available things in the static directory available without hitting the webapp Configure myapp/app.ini similarly to (taken from pylonshq site): [production] app_name = pylonshq runner = src/kai/production.ini version = 1 update_fetch = /sync_app service.couchdb = service.files = default_host = pylonshq.com In this case, the Pylons site needs CouchDB setup, and the files service (all Pylons sites should use the files service to store cached files, templates, etc.) $ toppcloud update myapp/ Note that we’re one directory above myapp, and we have a trailing slash on myapp, this is needed because an rsync is done to copy it to the server, don’t leave off the trailing slash! (This will prolly be fixed at some point) That’s it! 10 easy steps to go from zero to a running deployed website. I should also note that you will need to make a production.ini file, and that it should have a few important changes in it. All the references to %(here)s should be changed to %(CONFIG_FILES)s since that’s the persistent location that files can be stored for a toppcloud app between deployments. Other configuration information provided by services (Couch supplies the db/host, Postgres has its host/user/pass info) can be accessed via CONFIG_ vars as well. The services docs have some more info. More apps can be added to a host as desired, until the ram runs out of course. At the moment toppcloud is using a process pool of 5 with 1 thread under mod-wsgi for each application. This can use a bit of ram if you have multiple heavy Pylons processes, hopefully there will shortly be a way to ask toppcloud to use a single process with multiple threads which will help cut the ram profile a bit. Using Django and PHP Unfortunately I haven’t actually tried this myself, but there’s nothing preventing it. You’ll need to change the app.ini so that instead of using ‘src/kai/production.ini’ as the runner, it uses a Python file in the directory, say main.py that then loads the Django app as a WSGI application and returns it. Sort of like this. Note that the config vars Django needs for its database should then be present in os.environ when setting up the settings.py that Django uses. If you’re looking through the toppcloud source code by now, you may have also noticed there’s an example app that uses PHP. There’s nothing holding back toppcloud from setting up mod_passenger and deploying Ruby apps at some point either should someone wish to add that feature. dumb pipes There’s been numerous mentions on various blogs and in the news about how much the cell phone carriers hate the concept of being nothing more than “dumb pipes” for wireless Internet and phone use. That means of course, that they’d no longer be competing on what phones you could use but instead solely on service quality and price… I think the same type of transition is in store for shared hosting providers and some of the boutique app deployment shops like heroku. It’ll take awhile, toppcloud is very rough right now. But when you can d/l a nice little open-source package, run it, choose your choice of cloud provider (based on price + quality!), have it automatically setup for you to deploy your apps to, then deploy apps with a single command… you’ve already done the vast majority of what a service like heroku does, except you could still modify toppcloud if there was something lacking you really needed. And it’s a reason like that, that toppcloud exists to begin with. Oh, and thanks Ian for writing this before I wasted more time making it myself. [Less]

OpenID + DB Authentication in Pylons is easy with RPX | Tony Landis: Tony gives a great walk through on how to get OpenID auth in a Pylons app with RPX. Nicely done.

Beaker 1.4 has now been released, and addresses several fairly important bugs. First, the full changelog: Fix bug with hmac on Python 2.4. Patch from toshio, closes ticket #2133 from the TurboGears2 Trac. Fix bug with occasional ValueError from ... [More] FileNamespaceManager.do_open. Fixes #10. Fixed bug with session files being saved despite being new and not saved. Fixed bug with CacheMiddleware overwriting configuration with default arguments despite prior setting. Fixed bug with SyntaxError not being caught properly in entry point discovery. Changed to using BlobProperty for Google Datastore. Added domain/path properties to the session. This allows one to dynamically set the cookie’s domain and/or path on the fly, which will then be set on the cookie for the session. Added support for cookie-based sessions in Jython via the JCE (Java Cryptography Extensions). Patch from Alex Grönholm. Update Beaker database extensions to work with SQLAlchemy 0.6 PostgreSQL, and Jython. Note that the beaker database extension now works on Jython, and the cookies for sessions can be set dynamically during a request (for sites that operate across multiple domains/sub-domains). Most importantly though, a bug in the import of the Google back-end has been fixed, which caused installation failures on Beaker 1.3.x. Docs can be found on the Beaker site. To upgrade your Beaker with easy_install: easy_install -U Beaker This release is also notable as the majority of the fixes were contributed by several web framework communities. Thanks for the patches! [Less]

I’ve now updated the blog software powering my blog, which is very long overdue. In the past, this blog was run off Typo, which apparently now hosts their home site off the github (its moved a dozen times in the past 3+ years). Typo always worked ... [More] moderately well for me, however, I found it sluggish (Prolly Rails there), and incredibly horrid on ram usage. It was not at all unusual to see it running past 700 megs of RAM after running for just a few weeks, which is a bit annoying as the machine only has 4GB total and is running quite a few things. After last weeks SF Python Meetup on MongoDB I figured it was time to get a little actual MongoDB usage under my belt. I also inadvertently implemented enough of the MovableType XMLRPC API as I didn’t want my app to be too extensive, just enough for me to post to my blog. So in the end, I had a small set of requirements for the replacement: Not be horribly slow Not take up huge amounts of RAM Retain all existing URL’s (It really annoys me when people break their old links) Compatible with my blog software (MovableType / MetaWeblog XMLRPC API) Not screw up the RSS/Atom feeds and cause Planet Python to show all my posts as if they were new (I’ve seen this happen to a few people on occasion) I wanted to build it myself, because of course, the world definitely needs more blog apps, and I wanted one that used MongoDB. So for those curious, here’s the source code to the blog app. It’s rather rough, as its fairly custom built just for my needs, nor do I have any plans to expand it into some general purpose blog engine, with themes, etc. The only other thing pending at the moment is to add the ability to comment again, as I haven’t quite gotten that feature in yet. For those trying it out, the README should help get started, but its very rough (thus the name of the package). Strings, Unidecode, and Slugs When copying some functionality I needed from the Rails app (to retain URL compatibility), I noticed two things it did which I thought was handy. To convert a title into slug for the article, it used a fairly sophisticated scheme relying on two other packages. First, was the use of a Ruby port of a Perl package called Text::Unidecode which is pretty cool, and converts UTF-8 chars into their closest ASCII version. I figured someone must’ve ported it to Python as well, and sure enough, someone did! It wasn’t on the cheeseshop though, which was unbearable for me, so I’ve posted it to the Cheeseshop so others can easy_install it. Next up, was a Ruby library called stringex, which add’s a few things to Rails, including a string method called ‘to_url’. That method does a variety of transformations to remove all those funny characters from a title, and do a bunch of other neat changes of common characters to human readable versions (source for those conversions). I ported the key module of stringex to Python, and it resides in my blog app. If someone would like to extract it and make it into its own package, or even better, if I somehow missed the fact that someone else has ported it already, let me know (tweet me @benbangert!). I’ll be writing up my thoughts on making a small app with MongoDB, and how it differs from my experience working with CouchDB for PylonsHQ in a later post for those curious. [Less]