Zope

import logging from Products.CMFCore.utils import getToolByName from Products.PortalTransforms.Transform import make_config_persistent logger = logging.getLogger('MY.PACKAGE.setuphandlers') def isNotThisProfile(context, marker_file): return ... [More] context.readDataFile(marker_file) is None def setup_portal_transforms(context): if isNotThisProfile(context, 'MY.PACKAGE-PROFILENAME.txt'): return logger.info('Updating portal_transform safe_html settings') tid = 'safe_html' pt = getToolByName(context, 'portal_transforms') if not tid in pt.objectIds(): return trans = pt[tid] tconfig = trans._config tconfig['class_blacklist'] = [] tconfig['nasty_tags'] = {'meta': '1'} tconfig['remove_javascript'] = 0 tconfig['stripped_attributes'] = ['lang', 'valign', 'halign', 'border', 'frame', 'rules', 'cellspacing', 'cellpadding', 'bgcolor'] tconfig['stripped_combinations'] = {} tconfig['style_whitelist'] = ['text-align', 'list-style-type', 'float'] tconfig['valid_tags'] = { 'code': '1', 'meter': '1', 'tbody': '1', 'style': '1', 'img': '0', 'title': '1', 'tt': '1', 'tr': '1', 'param': '1', 'li': '1', 'source': '1', 'tfoot': '1', 'th': '1', 'td': '1', 'dl': '1', 'blockquote': '1', 'big': '1', 'dd': '1', 'kbd': '1', 'dt': '1', 'p': '1', 'small': '1', 'output': '1', 'div': '1', 'em': '1', 'datalist': '1', 'hgroup': '1', 'video': '1', 'rt': '1', 'canvas': '1', 'rp': '1', 'sub': '1', 'bdo': '1', 'sup': '1', 'progress': '1', 'body': '1', 'acronym': '1', 'base': '0', 'br': '0', 'address': '1', 'article': '1', 'strong': '1', 'ol': '1', 'script': '1', 'caption': '1', 'dialog': '1', 'col': '1', 'h2': '1', 'h3': '1', 'h1': '1', 'h6': '1', 'h4': '1', 'h5': '1', 'header': '1', 'table': '1', 'span': '1', 'area': '0', 'mark': '1', 'dfn': '1', 'var': '1', 'cite': '1', 'thead': '1', 'head': '1', 'hr': '0', 'link': '1', 'ruby': '1', 'b': '1', 'colgroup': '1', 'keygen': '1', 'ul': '1', 'del': '1', 'iframe': '1', 'embed': '1', 'pre': '1', 'figure': '1', 'ins': '1', 'aside': '1', 'html': '1', 'nav': '1', 'details': '1', 'u': '1', 'samp': '1', 'map': '1', 'object': '1', 'a': '1', 'footer': '1', 'i': '1', 'q': '1', 'command': '1', 'time': '1', 'audio': '1', 'section': '1', 'abbr': '1'} make_config_persistent(tconfig) trans._p_changed = True trans.reload()         And create the File MY.PACKAGE/MYPROFILESDIR/PROFILENAME/MY.PACKAGE-PROFILENAME.txt, so that this import step is not run for any profile but just for this one. This HowTo is also available in the . [Less]

of our regular meetings. I is now writing a MSc thesis on... python and concurrency. means a computer is executing several things simultaniously. even interacting with eachother. There's a lot to talk about. Here's some. (OO API, modeled on ... [More] Java). People don't quite like threading in python because of the infamous GIL (Global Interpreter Lock). Only one thread is allowed to run python interpreted code at the same time. This is needed to make python thread-safe. So the interpreter itself only runs in one thread. The GIL is bad as there's no gain on multicore machines. The OS cannot schedule various python threads on several cores: python itself locks itself continously on one core and the rest of the python process has to wait even though cores are sitting completely idle. There's also a problem with responding quickly to events. On the other hand, the GIL is also good. It means you can optimise the hell out of it on a single core: lots of performance and safe to tweak. And, importantly, it is a safe environment for C extentions. And... there are other solutions There've been a few tries at removing the GIL, mostly unsuccesful. The current policy for cPython is that a patch that removes it gets accepted if the speed on single processor machines is not slower module mimicks the module, only it runs in separate processes instead of separate threads. The invocation (apart from the import statement) is exactly the same. The end result is not completely the same. You don't have implicit shared memory, for instance. If you want to communicate between processes you need to do it explicity with inter-process communication. Asynchronous solutions. There are a couple of them. Asyncore, twisted, tornado. The basic pattern is always "don't call us, we'll call you". The one big gotcha is that . Your code is split in lots of small parts that are connected by callbacks and started by a central core ("reactor" in twisted). It is still a single CPU business, but the utilization is better. You can also have "hidden" async: eventlet, pyevent, gevent, epoll, kqueue, etc. They mostly react to kernel events ("yeah, I've read your full file, now you can have a go at it"). A core concept here is "greenlet". (Reinout: You'd better google for it instead of fully trusting my summary here). Kamaelia functions in a similar way, only by tieing everything to generators. And EVE online (a massive multiplayer game) uses "stackless python". Second life is where greenlets originated. tries to work with python in non-python environments. He's a cobol programmer and discovered he could generate quite a lot of cobol code with python which cut down on the amount of medial work. Another area was the batch language called JCL. He wrote a tool for it that converts from a visual JCL representation to the actual batch commands. wanted to know what really happened when he decorated a class's method with the class MyClass @property def my_method(self): return 'bla' It isn't an attribute or so. It turns out to be a special class attribute: a so-called method (and optionally method, too (which returns a bound method). is still studying on computer vision and python. is extracting information from images, usually with some sort of predefined purpose ("is someone smiling in this image?", "what is the number on the license plate?"). Information extraction uses several techniques. Color space conversion (RGB to Hue Saturation Value and only looking at hues, for instance). Edge detection. Line detection. Clustering. Some usecases are character recognition, handwriting recognition, surveillance, augmented reality. A practical example is the Dutch section control () where they need to find the car in images and then need to find the license plate, then color normalization, then you need to find the characters, then... There's a from a microsoft researcher on the generic computer vision subject. originated at Intel: an open source computer vision library with a python binding. The late 2009 2.0 release is pretty good. It includes a "ML" machine learning module for dealing with the fuzzy data that comes out of the computer vision part. python program that started up his webcam and did edge detection with him standing in front of his computer. Including straight line detection that picked up the straight edges of the projection screen, his glasses and the ceiling fixtures. Wow. On the whole, the installation procedure (unless you've got an official package) is hairy. The library itself is sometimes buggy. The mailinglist and irc channel is useless. And it is sometimes undocumented. But it is getting better. And... there's an O'Reilly book: lived in the stone age for a while as he didn't know about virtualenv two months ago. Virtualenv provides an isolated sandbox python environment so different installations don't step on eachother. Pip installs packages and allows you to install a fixed known set of them. An easy_install replacement. likes small tools and frameworks, so he's working a lot with small wsgi "frameworks". But you cannot have for instance the ZODB object database that way as it is too heavy. But he likes object database. . A simple codebase of just 500 lines. Everything is stored on disk as directories and text files. for debugging: you can use standard "cat", "ls", "grep" etc. Basic python datatype attributes are stored as plain text files, everything else is a python pickle. Pydiritems (so: the dictionary-like tree of objects) are directories. It loads fast enough to render it usable for cgi and wsgi. It is simple and effective. Quite stable. He uses it in production on a website with some 20000 objects with 5 or 6 attributes each. He's planning to release it Real Soon Now. Just one small modification left to make... [Less]

Sometimes when you are doing a lot of Plone development and integrations you could miss the big picture: Plone is not just a CMS. It's a damn good CMS with almost unlimited possibilities of integration. However its 'unlimity' has started to be one of its biggest limitations.

eGenix is pleased to announce mxODBC Zope DA version 2.0, our new Zope/Plone ODBC Database Adapter, with support for Zope 2.12 and Python 2.5 - 2.6 on all major platforms.

In a recent project we need to provide different roles to users, basing this choice to host name used to reach the Plone site

So thought I'd do a quick update of what we've been up to in the past couple of weeks, as things have been pretty busy here at Netsight. First thing to mention is that whilst we are primarily known for our Plone work, the three largest projects ... [More] that we are working on at the moment are not based on Plone. Aha! the doomsayers say, Plone is declining etc etc. But in fact the opposite, this is a positive thing... let me explain: One of the main criticisms of Zope and Plone back when they started was that they were not very 'pythonic' ie. everything in Zope and Plone was done differently in every other python system. Some would say that Zope was so ahead of it's time that there wasn't anything else out there, so it was blazing its own trail. Zope 2 was a very monolithic system and you basically had to use the entire stack. Not only that but every part of the stack was different to any other stack out there in the world. It was a big learning curve. The 'Z' shaped learning curve. Plone came along and hid a lot of this complexity to the average user, and simple customisation could be done fairly simply. As Plone got bigger, things got more complex, and some really clever stuff to make core developers lives easier made some simple customisations more tricky. This is now being addressed, and for Plone 4 much is being removed, creating a much sleeker system. However the biggest wins for the likes of companies like Netsight that have to recruit, train and retain developers is that there is much more transferable code and skills between Zope and Plone and the rest of the python world. Since much of Plone nowadays is developed in the much more flexible Zope 3 style using Component Architecture you can transfer between other frameworks like Grok repoze.bfg, vudo... and not just the Zope based ones... but its now much easier to move back and forth with WSGI based frameworks like Werkzeug, Pylons, etc. This is such a tangible benefit for a business. We have recently hired a new developer to work on one of the projects below, and we are . What is great, is we can look for just 'python' developers, not specifically 'plone' developers as they don't need to know 'plone' now like they used to in order to be productive in a Plone environment. To give some examples of this, let me talk about a few really interesting projects Netsight is working on at the moment... none of which directly involve Plone, but which all use skills transferable to and from modern Plone development. A no doubt Plone will be stirred into the mix at some point in both these projects' near future. The first is a several month support and development contract with a large academic-related business. They employ 1,500 staff and their business is interacting with teachers and students across the globe. They built (using internal people) several Zope based web systems for managing the interactions with these teachers and students. These were built quite a few years ago, and are in need of a bit of a refresh and can do with moving to Zope 3. With the help of another company they have moved most of the code out of the ZODB and onto the filesystem so it can me managed much easier. Our first task is to build an events management system for them. They run events worldwide, and this system will take care of bookings and related tasks. What makes this project so interesting is that we are able to now develop using Zope 3 style and technologies via Five on a Zope 2 existing system. This is really nice as it gives us a much nicer development model (classes, views, adapters etc) rather than the old style of a bunch of 'Script (Python)' objects. It makes it much easier for us to debug (via pdb) and to benchmark and test. Not only that, but we have a new developer working on the project, who whilst has python experience, has little specific Zope experience. Working in this way he has been able to concentrate on the pythonic aspects and leave the core Zope tasks to the more experienced developer on the project. The second project is really quite a fun challenge. We are dynamically re-skinning an existing .NET portal site... without touching the original site. We are using a WSGI stack with modules that we are developing based upon xdv to dynamically clean up the (nasty, broken) HTML from the original portal and present a nice clean, reskinned site to the end user. Oh, and as we are gluttons for punishment, we are re-writing the URLs of the site too... [Less]

The way form data submission is handled in web apps has irritated me for many years now. Thankfully, I think we may be able to make it better.

---

Next to the method for defining content type schema and installing content types in a portal, the reference engine has been one of the more valuable and widely used bits of Archetypes. Unfortunately, the AT reference engine is also a bit heavy ... [More] weight and too tightly coupled to AT, making it hard to re-use fruitfully. package provides a very powerful library for establishing connections, read references and relationships, between objects. It also provides for walking and searching networks of connections. It's all very cool. It's also very generalized which is a good thing since it can be used in so many different ways. This does, however, make it very difficult to approach. It's my hope that as we find ways to integrate zc.relation into the Plone stack, that others will find clever ways to use zc.relation's powerful network traversing to do some unexpected things. So the combination of zc.relation and provides not only a more widely re-usable, loosely coupled replacement for the AT reference engine, but a much more powerful tool with capabilities far beyond what could ever be achieved with the AT reference engine. One of the concerns which kept coming up was that getting rid of the AT reference engine meant getting rid of UIDs. With use taking off and finding new and interesting uses it's important that we provide some sort of UUID system, but the AT reference engine is large and heavy and very much unnecessary to support UUIDs. UUIDs are very simple and should not be coupled to any particular framework or larger library when they can so easily be a very small library of their own easily integrated in lots of ways. So the question of how we want to support UUIDs and the question of what to do with the AT reference engine should no be conflated, they can and should be addressed separately. integrates the z3c.relationfield package to provide widgets for specifying zc.relation connections between content on z3c.form forms. This allows Dexterity content to establish relationships and references to both Dexterity and AT content. There is, however, no zc.relation field and widget for AT content so there's no way for AT content to establish relationships or references to Dexterity content. set out to build an AT field and widget that could make use of zc.relation relations and on the latter half of the second day I joined him for some pair programming. By lunch on the third day, we had working extensions of Products.Archetypes.Field.ReferenceField and Products.Archetypes.Widget.ReferenceWidget which used zc.relation behind the scenes. Next up is an extension of Products.ATReferenceBrowserWidget which works with zc.relation. Once that is working, can be used with one which uses zc.relation. Finally, a migration can be to offer a ZCML file which would replace the ATCT relatedItems field implemented to migrate existing AT references to the zc.relation back-end. Once that is completed archetypes.z3crelationfield can be used as a proving ground for the possibility of removing the reference engine from Archetypes itself. Once it's polished and the AT reference engine removed, the way will be paved to remove one of the hurdles which has made it so hard for newer content type frameworks to make an entrance on the Plone scene. [Less]

Next to the method for defining content type schema and installing content types in a portal, the reference engine has been one of the more valuable and widely used bits of Archetypes. Unfortunately, the AT reference engine is also a bit heavy ... [More] weight and too tightly coupled to AT, making it hard to re-use fruitfully. package provides a very powerful library for establishing connections, read references and relationships, between objects. It also provides for walking and searching networks of connections. It's all very cool. It's also very generalized which is a good thing since it can be used in so many different ways. This does, however, make it very difficult to approach. It's my hope that as we find ways to integrate zc.relation into the Plone stack, that others will find clever ways to use zc.relation's powerful network traversing to do some unexpected things. So the combination of zc.relation and provides not only a more widely re-usable, loosely coupled replacement for the AT reference engine, but a much more powerful tool with capabilities far beyond what could ever be achieved with the AT reference engine. One of the concerns which kept coming up was that getting rid of the AT reference engine meant getting rid of UIDs. With use taking off and finding new and interesting uses it's important that we provide some sort of UUID system, but the AT reference engine is large and heavy and very much unnecessary to support UUIDs. UUIDs are very simple and should not be coupled to any particular framework or larger library when they can so easily be a very small library of their own easily integrated in lots of ways. So the question of how we want to support UUIDs and the question of what to do with the AT reference engine should no be conflated, they can and should be addressed separately. integrates the z3c.relationfield package to provide widgets for specifying zc.relation connections between content on z3c.form forms. This allows Dexterity content to establish relationships and references to both Dexterity and AT content. There is, however, no zc.relation field and widget for AT content so there's no way for AT content to establish relationships or references to Dexterity content. set out to build an AT field and widget that could make use of zc.relation relations and on the latter half of the second day I joined him for some pair programming. By lunch on the third day, we had working extensions of Products.Archetypes.Field.ReferenceField and Products.Archetypes.Widget.ReferenceWidget which used zc.relation behind the scenes. Next up is an extension of Products.ATReferenceBrowserWidget which works with zc.relation. Once that is working, can be used with one which uses zc.relation. Finally, a migration can be to offer a ZCML file which would replace the ATCT relatedItems field implemented to migrate existing AT references to the zc.relation back-end. Once that is completed archetypes.z3crelationfield can be used as a proving ground for the possibility of removing the reference engine from Archetypes itself. Once it's polished and the AT reference engine removed, the way will be paved to remove one of the hurdles which has made it so hard for newer content type frameworks to make an entrance on the Plone scene. [Less]