Turnkey Linux

I just finished upgrading the WYSIWYG editor on the web site from FCKEditor to CKEditor, which is FCKEditor's less offensively named successor. Improvements A noticeably faster, more lightweight code-base which was rewritten from ... [More] the ground up. Full-screen mode works beautifully now. I love how this instantly removes all the clutter and lets me focus on the content I'm editing without being distracted by the rest of the web site. The edit area can be resized dynamically by dragging the window corner, giving you as much room to edit as you need. If you've ever felt uncomfortably cramped inside a tiny edit box you know what a relief it can be to escape. Source mode now pretty prints your XHTML tags, with proper indenting. When you need to edit raw source, this feature makes it much easier to visually parse what you're looking at. A status field provides information showing your current position in the document's tag hierarchy. Advanced theming support. The new default theme Kama let's you pick a colorset to match the theme of your web site: Personally I like Kama better but if you're feeling attached to FCKeditor's somewhat clunky looks you can select CKEditor's V2 theme which looks just like it: Out of the box CKEditor also comes with a rather pretty Office 2003 theme: Drupal installation and upgrade notes On Drupal, you install CKEditor the same way you install FCKEditor. First you download CKEditor and the CKEditor Drupal module. Next, extract the CKEditor Drupal module in your site modules directory and then extract CKEditor itself into the module's directory, like this: cd /etc/drupal/6/sites/all/modules tar -zxvf ckeditor-6*.tar.gz mv ckeditor_3* ckeditor cd ckeditor tar -zxvf ckeditor_3* Enable the CKEditor module while disabling FCKEditor (I.e., enabling both will result in Javascript errors). Drupal's CKEditor module inherits the settings from the FCKEditor module so the upgrade is relatively painless. If you've customized FCKEditor's button layout by editing modules/fckeditor/fckeditor.config.js you'll want to customize CKEditor's button layout as well by editing the profiles in modules/ckeditor/ckeditor.config.js. Most of the button names are the same except for a handful which have been renamed: FormatName => Format FontFormat => Font UnorderedList => BulletedList OrderedList => NumberedList StrikeThrough => Strike Rule => HorizontalRule Gotcha: pop-up mode is broken (but good riddance!) FCKEditor was slow enough that we decided to disable it by default for administrator accounts on this web site while still allowing us to launch it on demand in a pop-up when we really needed it. The bad news is that pop-up mode doesn't work well in the version of CKEditor I've been testing so I've had to disable it. The good news is that CKEditor is fast enough now that I can enable it by default and the full-screen mode works even better for minimizing distractions. Come to think of it, pop-up mode in FCKEditor was kind of a disaster. If you accidentally tried opening more than one editing pop-up, you would loose loose your previous edit without warning. After that happened a few times I developed a deep mistrust for FCKeditor and would edit the bulk of my content without it. Good riddance to that! [Less]

Keeping it simple, HTTPS is a combination of the HTTP and SSL/TLS protocols, which provides encryption and identification of the server.  The main idea is to create a secure channel over an insecure network, ensuring "reasonable" protection from ... [More] eavesdroppers and man-in-the-middle attacks. The trust inherent in HTTPS is based on the inclusion of "trusted" Certificate Authority (CA) certificates pre-installed in web browsers.  Basically, if your SSL certificate is not signed by one of these CA's, the browser will display a warning. TurnKey appliances generate self signed certificates on firstboot to provide an encrypted traffic channel, but because the certificates are not signed by a trusted CA, the warning is displayed. In most cases, this is acceptable. If it's not, go get an signed certificate. Authoritatively signed certificates Cost Authoritatively signed certificates can be costly, for example, Verisign (the most well known CA) charges $1,499 per year for their recommended certificate. There are cheap alternatives (I recently purchased a certificate from Go Daddy for $12.99) as well as a couple of free providers. Generate key and CSR First up is to create a certificate key and a certificate signing request (CSR). This can be done with OpenSSL. apt-get update apt-get install openssl # replace bold type with your info openssl req -new -newkey rsa:2048 -nodes -out www_example_com.csr -keyout www_example_com.key -subj "/C=US/ST=Arizona/L=Scottsdale/O=Example Company Inc./CN=www.example.com" Submit the CSR The above will generate two files, www_example_com.key and www_example.com.csr. Once you have signed up for an authoritatively signed certificate, you will be requested to upload the CSR file or its contents. Verify the request The signing authority will need to verify the validity of the request and that it was submitted by the entity to which the domain in the request is registered, usually done by contacting the administrative contact for the domain. Further steps may be required when requesting an Extended Validation (EV) certificate, which color the address bar green in recent browsers. Download signed certificate After validation, your signed certificate (crt) will be available for download. Most likely your signing authority will include an intermediate CA certificate bundle (trust chain). Note: you should make a backup of all SSL related files. Generate PEM and placement Generate the pem from the key and crt cat www_example_com.key www.example.com.crt > cert.pem Place the generated pem and intermediate bundle (eg. bundle.crt) in /etc/ssl/certs/, and make them read-only to root. chown root:root *.pem *.crt chmod 400 *pem *.crt Update configuration, enable SSL and reload webserver Apache configuration <VirtualHost *:443> SSLEngine on SSLCertificateFile /etc/ssl/certs/cert.pem SSLCertificateChainFile /etc/ssl/certs/bundle.crt </VirtualHost> a2enmod ssl /etc/init.d/apache2 force-reload Lighttpd configuration /etc/lighttpd/conf-available/10-ssl.conf $SERVER["socket"] == "0.0.0.0:443" {     ssl.engine  = "enable"     ssl.pemfile = "/etc/ssl/certs/cert.pem"     ssl.ca-file = "/etc/ssl/certs/bundle.crt" } lighty-enable-mod ssl /etc/init.d/lighttpd force-reload Do you use an authoritatively signed certificate? Is self-signed sufficient? Leave a comment! [Less]

Psssst. Hey you... yeah you. Word on the street is your shell scripts don't do any error handling. They just chug happily along even when everything is broken. Because a lowly shell shell script doesn't need any error handling right? WRONG! ... [More] <!--break--> Here are two simple tricks that are easy to use and will make your scripts much more robust. Turn on -e mode (do you feel lucky - punk?) In this mode any command your script runs which returns a non-zero exitcode - an error in the world of shell - will cause your script to itself terminate immediately with an error. You can do that in your shebang line: #!/bin/sh -e Or using set: set -e Yes, this is what you want. A neat predictable failure is infinitely better than a noisy unreliable failure. If you REALLY want to ignore an error, be explicit about it: # I don't care if evil-broken-command fails evil-broken-command || true Oh and as long as you're messing with shell modes, -e goes well with -x (which I like to think of as shell X-ray). Like this: #!/bin/sh -ex Or like this: # turn -x on if DEBUG is set to a non-empty string [ -n "$DEBUG" ] && set -x That way you can actually see what your script was doing right before it failed. Use trap for robust clean-ups A trap is a snippet of code that the shell executes when it exits or receives a signal. For example, pressing CTRL-C in the terminal where the script is running generates the INT signal. killing the process by default generates a TERM (I.e., terminate) signal. I find traps most useful for making sure my scripts clean-up after themselves whatever happens (e.g., a non-zero error code in -e mode). For example: #!/bin/sh -e TMPFILE=$(tempfile) trap 'echo "removing $TMPFILE"; rm -f $TMPFILE' INT TERM EXIT echo TMPFILE=$TMPFILE echo hello world > $TMPFILE cat $TMPFILE # gives user a chance to press CTRL-C sleep 3 # false always returns an error false echo "NEVER REACHED" Note that you can only set one trap per signal. If you set a new trap you're implicitly disabling the old one. You can also disable a trap by specifying - as the argument, like this: trap - INT TERM EXIT [Less]

Open source virtualization has been evolving dramatically over the last few years. Incumbent proprietary platforms such as VMWare still hold the throne in many areas but open source competitors are gaining ground fast on their way to ubiquity. Things ... [More] are a-changing in the land of virtualization. Right now we have three contenders to the (open source) throne battling it out for supremacy: Xen: backed by Citrix VirtualBox: backed by Oracle/Sun KVM: backed by RedHat <!--break--> Neither Citrix nor Oracle have yet to establish a good track record with regards to open source so I recently took the time to dig a little deeper into KVM and qemu, two projects which I suspect are going to be a huge part of the future of open source virtualization. KVM: at its essence KVM is just a kernel infrastructure. It's not a product. The developers forked qemu to hook into the KVM kernel infrastructure and take advantage of it. Eventually the fork was merged back into the qemu mainstream. qemu: powers nearly all Linux virtualization technologies behind the scenes. Qemu is a spectacular piece of software written largely by Fabrice Bellard, a programming genius who also wrote TCC (tiny c compiler), FFMpeg and many other less famous programs. Highlights from my findings: I'm pretty sure KVM is the future of open source virtualization. Not Xen or VirtualBox. KVM is a latecomer to the virtualization game but its technical approach is superior and many believe it will win out eventually over other virtualization technologies and become the transparent virtualization standard of the future. Personally, I think that's a pretty likely outcome at this point. The primary advantages going for KVM is simplicity and leverage. By taking advantage of the hardware-level support in new processors, KVM can be many times simpler than competing technologies such as Xen or VMware while achieving equivalent or superior performance. KVM leverages the Linux kernel in a way that allows its developers to focus their resources more efficiently on actual virtualization related development. The proof is in the pudding. With just a fraction of the resources their competitors have, the KVM implemented advanced features such as live migration, and SMP support (e.g., up to 255 virtual CPUs regardless of how many CPUs the host has). Even recursive virtualization should be possible (I know patches were written to get it to work). libvirt: Redhat are spearheading development of a backend-agnostic management layer for virtualization (libvirt). The fundamental design is pretty good, but for our purposes (I.e., TurnKey development) libvirt and its higher-level friends just get in the way. The really useful parts are the qemu-based primitives, which are easier to use directly than through the libvirt stuff. qemu supports multiple mechanisms for plugging guest NICs together - via a TCP/UDP port, attaching to a tap/tun device or even plugging into VDE (virtual distributed ethernet), which allows arbitrarily complex virtual network topologies to be constructed that span multiple physical machines. I've played around with a few test configuration and it's pretty neat. Qemu vs VirtualBox Though it's hard to tell, VirtualBox was forked from an earlier version of qemu. The main thing VirtualBox added was sophisticated binary recompilation that allowed fast virtualization without requiring support from the underlying hardware. Today most modern PC processors from Intel and AMD have hardware support for virtualization which makes binary recompilation techniques unnecessary. This allows hypervisors such as KVM to achieve the same results using a simpler hardware-enabled approach with superior performance. VirtualBox also added a user-friendly GUI shell that simplifies setting up virtual machines in a desktop environment. In general, I was favorably surprised by qemu. If VirtualBox is the notepad of open source virtualization. Qemu is Vim. In other words if you're looking for a swiss-army virtualization knife you can get hacking at, qemu beats VirtualBox many times over. Random insights: Qemu has equivalent-better performance with KVM If your hardware can't support KVM, then performance won't be as good as VirtualBox but if you use the KQEMU kernel acceleration module, it will be acceptable (e.g., on my system 30 seconds with KQEMU to boot lamp vs 20 seconds in KVM node) Qemu is a better primitive than VirtualBox: Good defaults are chosen if you don't specify otherwise It's simpler to launch VMs from the cli: qemu -cdrom appliance.iso # install appliance.iso to test.img qemu-img create /virtdisks/test.img 4G qemu -cdrom appliance.iso -hda /virtdisks/test.img Supports launching VMs as detached daemon processes Supports VNC graphics mode with optional encryption and authentication Easier to script Supports a scriptable VM monitor Can work without root privileges, unless you want to use tap/tun interfaces Also, it wouldn't be too complicated to write a wrapper that allows users to safely run qemu + tap/tun without root privileges. Easy to use default usermode networking built-in NAT, dhcp, tftp host can be accessed by guest on 10.0.2.2 supports forwarding host ports to guest (-redir option) Qemu can boot 64bit guests on a 32bit host using processor emulation. I tried this and it works, though its pretty slow: qemu-system-x86_64 -cdrom ./lenny-amd64-standard.iso Qemu can be configured to allow transparent chrooting into non-native chroots without launching a VM (e.g., chroot into a 64bit debootstrap from a 32bit system) Its a bit tricky and the default Ubuntu package doesn't yet include the Debian patches that enable this very neat trick. For usage scenarios where cli usage isn't optimal, you can use one of several GUI front-ends written for qemu. I liked qemuctl and qemu-launcher (for different purposes).   [Less]

Each Amazon EC2 instance has associated metadata, as well as user data supplied when launching the instance. The meta and user data is instance-specific, and therefore only accessible to the instance. The data is useful on several levels, such as ... [More] configuring SSH public keys, programmatically configuring the instance according to certain criteria, or even executing user supplied initialization scripts.   Retrieving the data Retrieving the data is done by querying an Amazon web server with the base URI of http://169.254.169.254/API-VERSION. The available API versions can be queried by performing a GET request on http://169.254.169.254/. The latest version of the API is always available using the URI http://169.254.169.254/latest. There is quite a lot of information available through the API, some more useful than others. For example, ami-id, ami-launch-index, availability-zone, instance-id, public-ipv4, user-data, ... (see below for the full list). Some notes on user data One of the most useful pieces of data is user-data, which can be used to pass configuration information or even initialization scripts to the instance upon launch. User data must be base64 encoded, and is limited to 16k (pre-encoding).  The popular API tools usually handle the encoding transparently, so you shouldn't have to worry about it. The data is also decoded before presented to the instance, so again, you shouldn't need to worry. What you do need to worry about though, or at least be aware of, is security. The user-data (and all metadata for that matter) can be accessed by any user or process on the instance. So please, please, do not specify any secret information in user-data unless you are absolutely sure what you are doing. Even then, I'd think twice. But, you say, I trust all my users and processes. OK, how about this (thanks go to Eric Hammond for this example). You run a website that allows users to upload files by specifying a URL. The user specifies http://169.254.169.254/latest/user-data, and lo-and-behold, your user-data and any secrets included have been divulged. Do you still want to include secrets in user-data? The simple way The simplest way of retrieving metadata is by use of a command line network tool, such as curl, for example: curl http://169.254.169.254/latest/meta-data/public-ipv4 The more programmatic way Usually you need a more programmatic type interface, and there are a couple of libraries for different languages available. I didn't find one that met my needs, so I wrote one in Python called ec2metadata.py. I licensed the copyright over to Canonical so it could be included in Ubuntu's ec2-init package. ec2metadata.py has a CLI interface, as well as a Pythonic interface: $ ec2metadata.py # all options will be displayed $ ec2metadata.py --instance-id # displays the instance id import ec2metadata instanceid = ec2metadata.get('instance-id') print instanceid It can be very useful when coupled with inithooks, for example, setting of the SSH public keys on first boot. #!/usr/bin/python # # Query and display EC2 metadata related to the AMI instance # Copyright (c) 2009 Canonical Ltd. (Canonical Contributor Agreement 2.5) # # Author: Alon Swartz <[email protected]> # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see <http://www.gnu.org/licenses/>. """ Query and display EC2 metadata If no options are provided, all options will be displayed Options: -h --help show this help --kernel-id display the kernel id --ramdisk-id display the ramdisk id --reservation-id display the reservation id --ami-id display the ami id --ami-launch-index display the ami launch index --ami-manifest-path display the ami manifest path --ancestor-ami-id display the ami ancestor id --product-codes display the ami associated product codes --availability-zone display the ami placement zone --instance-id display the instance id --instance-type display the instance type --local-hostname display the local hostname --public-hostname display the public hostname --local-ipv4 display the local ipv4 ip address --public-ipv4 display the public ipv4 ip address --block-device-mapping display the block device id --security-groups display the security groups --public-keys display the openssh public keys --user-data display the user data (not actually metadata) """ import sys import time import getopt import urllib import socket METAOPTS = ['ami-id', 'ami-launch-index', 'ami-manifest-path', 'ancestor-ami-id', 'availability-zone', 'block-device-mapping', 'instance-id', 'instance-type', 'local-hostname', 'local-ipv4', 'kernel-id', 'product-codes', 'public-hostname', 'public-ipv4', 'public-keys', 'ramdisk-id', 'reserveration-id', 'security-groups', 'user-data'] class Error(Exception): pass class EC2Metadata: """Class for querying metadata from EC2""" def __init__(self, addr='169.254.169.254', api='2008-02-01'): self.addr = addr self.api = api if not self._test_connectivity(self.addr, 80): raise Error("could not establish connection to: %s" % self.addr) @staticmethod def _test_connectivity(addr, port): for i in range(6): s = socket.socket() try: s.connect((addr, port)) s.close() return True except socket.error, e: time.sleep(1) return False def _get(self, uri): url = 'http://%s/%s/%s/' % (self.addr, self.api, uri) value = urllib.urlopen(url).read() if "404 - Not Found" in value: return None return value def get(self, metaopt): """return value of metaopt""" if metaopt not in METAOPTS: raise Error('unknown metaopt', metaopt, METAOPTS) if metaopt == 'availability-zone': return self._get('meta-data/placement/availability-zone') if metaopt == 'public-keys': data = self._get('meta-data/public-keys') keyids = [ line.split('=')[0] for line in data.splitlines() ] public_keys = [] for keyid in keyids: uri = 'meta-data/public-keys/%d/openssh-key' % int(keyid) public_keys.append(self._get(uri).rstrip()) return public_keys if metaopt == 'user-data': return self._get('user-data') return self._get('meta-data/' metaopt) def get(metaopt): """primitive: return value of metaopt""" m = EC2Metadata() return m.get(metaopt) def display(metaopts, prefix=False): """primitive: display metaopts (list) values with optional prefix""" m = EC2Metadata() for metaopt in metaopts: value = m.get(metaopt) if not value: value = "unavailable" if prefix: print "%s: %s" % (metaopt, value) else: print value def usage(s=None): """display usage and exit""" if s: print >> sys.stderr, "Error:", s print >> sys.stderr, "Syntax: %s [options]" % sys.argv[0] print >> sys.stderr, __doc__ sys.exit(1) def main(): """handle cli options""" try: getopt_metaopts = METAOPTS[:] getopt_metaopts.append('help') opts, args = getopt.gnu_getopt(sys.argv[1:], "h", getopt_metaopts) except getopt.GetoptError, e: usage(e) if len(opts) == 0: display(METAOPTS, prefix=True) return metaopts = [] for opt, val in opts: if opt in ('-h', '--help'): usage() metaopts.append(opt.replace('--', '')) display(metaopts) if __name__ == "__main__": main()   [Less]

While reading CSS books and experimenting I found it helpful to take notes and put together a cheat sheet for quick reference. I figured others might find it useful so I'm sharing. I've also uploaded a simple PDF version. <!--break--> ... [More] Selectors Basic h1 select all <h1> tags h1.special select h1 tags of the special class h1#special select h1 tags of special id p a select <a> tags descendant from p h1, h2 select both h1, h2 * select all tags implicit in .class = *.class #class = *#class #banner * select all tags inside banner a:link selects link that guest hasn't visited regular style a:visited link that visitor has clicked before a:hover link when mouse is hovered over a:active link when it's clicked (just a ms) .class1.class2 select elements that have both class1 and class2 Pseudo-classes generate content :before inserts content before p.tip:before {content: "HOT TIP!" } :after like :before but after properties generated pseudo-classes can be styled like any other display color border content string content: "(link) " urls content: url(path/to/image) attributes content: attr(href) combinations content: " [" attr(href) "]" :first-child select and format just the first child of an element e.g., li:first-child { font-weight: bold; } formats all first elements in bold :focus when element receives focus (e.g., click or tab) e.g., input:focus { background-color: #FFFFCC; } Advanced selectors child selectors body > h1 select any h1 that is a child of body adjacent siblings h2 + p select p right after h2 attribute selectos img[title] select only img tags with "title" attribute .photo[title] select any element with photo class and a title attribute input[type="text"] input fields of type text a[href][title] select a that has both href and title attrs *[title] any element that has title attribute img[title~="Figure"] any img with title that contains Figure in it regexp match? img[title^="bar"] any img with title that starts with "bar" img[title$="bar"] any img with title that ends with "bar" img[title*="bar"] any img with title that contains substring "bar" *[lang|="en"] any element with lang attr that equals en begins with en- Pseudo-classes :first-line :first-letter just the first letter can only be applied to block elements (not inline) Psuedo-elements :before :after # examples h2:before {content: "]]"; color: silver;} body:after {content: " The End.";} Floats containing block nearest block-level ancestor floated element generates a block box regardless of its type will be laid out as a float Rules stay within borders of containing element left/right outer edge may not be to the left of inner left/right containing block. give precedence to existing floats prevent floats from overwriting each other floats are safe left outer edge must be to the right of the right outer edge of a preceding element. no overlapping between float elements if they collide the loosing element will be floated down can't be higher than inner top (pre-padding) of containing parent floats can't be higher than the tops of preceding floats If we have 3 floats, and the first two are floated to the left, the third float will float only as high as the second float. floats can't float higher than the top of the line box generated by a preceding element floats get pushed down to a new line height if there isn't room for them in the containing box given the above constraints, float as high as possible left floating elements try to float as left as possible, right floating elements try to float as right as possible BUT a higher position is preferred to floating more right Rule consequences when floated element is taller than container? the bottom sticks out of the container countermeasure floated element will expand to contain floated descendants negative margins can cause floats to appear to escape their parents just like negative margins on non-floated elements can make the child appear wider than it's parent if floated element is wider than it's parent it can stick out negative margins may cause a float to overlap inline elements inline elements rendered on top of the float a float expands to contain anything in it move to the left or right edge of the browser window (or containing window) floated inline methods are treated like blocks content with a background or border runs underneath the float how to prevent this - overflow: hidden clear property don't wrap around a floated item left - drop below left floated items right - drop below right floated items both - drop below both Positioning position := static | relative | absolute | fixed | inherit static generated as normal relative offset by some distance element retains it's shape space it would ordinarily have occupied preserved absolute removed from the flow positioned with respect to its *containing* block may be another element in the document space the element might have occupied is closed up as if the element didn't exist positioned element generates a block-level box even if it's inline fixed element box behaves like absolute but containing block is the viewport itself containing block in HTML the root element is HTML some browsers use body initial containing block rectangle the size of the viewport nearest ancestor (of any kind) that has position value != static block-level containing ancestor padding edge (I.e., border bounded area) inline-level containing ancestor content edge of the ancestor in ltr lang - top left corner, right bottom corner no ancestors? initial containing block important - elements can be positioned outside containing block maybe it should be called "positioning context" instead offset properties top, right, bottom, left := <length> | <percentage> | auto | inherit percentage containing width for left / right height for top / bottom auto element position if it were static positioning can cause height/width to be calculated automatically e.g., height 100% top: 0 bottom: 0 width 100% left: 0 right: 0 margins apply to the position boundaries you're really specifying the position of the outer edge setting width/height can make a difference IF you set borders, padding, margin because width/height set inner edges the content box fixed position can be used to simulate frames of old just like absolute position except the containing block is the viewport Properties formatting text line-height letter-spacing # how much space to add between letters font-family sans # clean and simple appearance (for headlines) Arial Helvetica Verdana Tahoma Formata Sans serif # better for long passages of text Georgia Times Times New Roman popular combinations Arial, Helvetica, sans-serif "Times New Roman", Times, serif "Courier New", Courier, monospace Georgia, "Times New Roman", Times, serif font size keywords xx-small x-small small medium large x-large xx-large each increase or decreases by 1.2 ems and percentages the same thing font styles font-style: italic|bold|normal text-transform: uppercase font-variant: small-caps text-decoration: underline|overline|line-through|blink spacing attributed letter-spacing word-spacing line-height normal setting is 120% alignment text-align: left|center|right|justify text-indent e.g., text-indent: 3em indent first line 3em text-shadow <color> <offset-x> <offset-y> <blur-radius>? text-shadow: green 5px 0.5em; not supported in Firefox white-space normal | nowrap | pre | pre-wrap | pre-line | inherit normal := discards extra whitespace collapses multiple "spaces" pre := treated as if it's pre whitespace is nog ignored pre-line := whitespace collapsed, linefeeds honored pre-wrap := like pre except lines wrap no-wrap := prevent wrap breaks from being rendered line breaks must be inserted with <br /> lists list-style-type: square|disc|circle list-style-position: outside|inside margins and padding margin: 5px 5px on all sides margin: 5px 1px 5px top and bottom 1px on the sides margins between elements don't get added the larger of the two margins is applied padding is added whenever vertical margins touch the margins collapse inline vs block display: inline|block; inline elements don't get any taller with padding or margins except for img tags colors rgb(30%, 30%, 30%) #FFF #FFFFFF names aqua fuchsia lime olive red white black gray maroon orange silver yellow blue green navy purple teal border each side can have a different property width / height em is the text size percentages = size of containing element max-width min-width max-height min-height width/height do not include borders, padding and margins just the content displayed width = width + left padding + right padding + left border + right border + left margin + right margins height property dangerous because it's hard to know in advance the size of content in a box amount of margins between paragraphs, headlines varies from browser to browser for consistent predictable results set this yourself background images background-image: url(images/bg.gif); url is relative to the stylesheet not the HTML page root relative absolute background-repeat repeat normal setting, repeat both on x and y repeat-x repeat-y no-repeat displays the image a single time background-position precise values <distance-from-left> <distance-from-top> keywords <horizontal> <vertical> <horizontal> := left|center|right <vertical> := top|center|bottom bottom is the bottom of the content not necessarily the bottom of the pag e.g., background-position: left center percentages short hand background := <background-color> <background-image> <background-attachment> <background-position> it's possible to replace borders with hand-drawn lines overflow := hidden | visible | scroll | auto visible - normal setting what browsers usually do scroll - add scroll bars auto - make scroll bars optional hidden - hides extended content clipping can provide precise control over hidden content overflows visibility := visible | hidden | collapse | inherit in invisible state the element is still there you just can't see it it effects layout normally descendant of invisible element can be visible Outlines Like borders except: they don't cause reflow: good for adding an outline to emphasize focus they can be non-rectangular they can invert the background Syntax: outline := <outline-color> || <outline-style> || <outline-width> outline-width := thin | medium | thick | <length> only one (unlike borders with top bottom left right) outline-style := dotted | dashed | solid | double | groove |ridge | inset | outset outline-color := <color> | invert | inherit Tips separation of concerns HTML should describe structure don't use class names that describe style CSS should describe styling start with an inline style sheet after perfection move CSS code to an external sheet title attribute adds tooltips to images and links don't use div when you can use HTML tags classes vs ids classes for repeating elements id selectors for unique elements ids get priority margin: 0 auto lets content float equivalent to <center> margins collapse line-height creates implicit padding Workflow put temporary code on top of the stylesheet makes it easiest to work with firebug name classes by structure, not styling separation of concerns HTML for content structure CSS for styling refactor eliminate code CSS HTML let the future take care of itself use advanced selectors (instead of tons of classes) use more tag types when available not everything should be a div minimize reloads edit CSS mode better than editing on server copy to server at the end refactor in gvim debugging draw borders to better understand box model use two firebug windows one for selecting one for editing live Misc element types replaced vs non-replaced whether or not the content is included in the HTML most html tags non-replaced replaced tags img input tag block vs inline block by default fills its parent element can not have other elements at its sides I.e., generates a break before and after box e.g., p div li special case that generates marker inline generated line boxes that are flowed within parent internal style sheet <style type="text/css"></style> only in the head inline style sheets no time and bandwidth saving class names letters, numbers, hyphens and underscores must start with a letter case sensitive terminology ancestor descendent parent closest ancestor child closest descendent siblings inheritance not passed border properties placement properties margins paddings Specificity of CSS properties highest specificity wins weights tag selector = 1 class selector = 10 ID selector = 100 inline style = 1000 overruling specifiy !important after any prperty a { color: teal !important; } organizing styles and stylesheets name styles by purpose not appearance refactor common styles to separate classes combine multiple classes group styles apply to related parts ofa page group styles with related purpose xhtml differences from HTML lowercase tags quotation marks required for xhtml all tags must be closed <br /> validator.w3.org tags p # for paragraphs li dl # definition list dt # definition term blockquote # quotes q # one line quotes cite # referencing address # identify and supply contact info table tags caption attribute align=top|bottom colgroup col thead tr th tbody tr td cell properties text-align vertical-align top|baseline|middle|bottom border-collapse: collapse|separate [Less]

Recently I needed to transfer data between entities, but I needed to keep the data secure from prying eyes, and its integrity intact from busy little fingers on the wire. I needed the solution to be simple, and support a high-performance ... [More] environment. Seeing that I could exchange a secret key over a secure channel out-of-band (OOB), I opted for using symmetric-key cryptography. Let me start off by saying that cryptography is a vast, fascinating and complex subject. I'll discuss some of the high-level key concepts related to the subject matter to provide some background, but thats about it. If you're interested, I recommend the above link as well as Applied Cryptography by Bruce Schneier.   What is symmetric encryption (and asymmetric for that matter) In a nutshell, symmetric-key cryptography refers to encryption methods in which both the sender and the receiver share the same key. This requires establishing a secure channel for secret key exchange, which also presents a considerable and practical chicken-and-egg problem. This is where asymmetric-key (or public-key) cryptography comes in.  Whitfield Diffie and Martin Hellman first proposed the notion of public-key cryptography in 1976, in which two different but mathematically interrelated keys (public and private) are used. The public key (freely distributed) is typically used for encryption, while the private key is used for decryption. Enough background for now, lets get to it!   Choosing the crypto library In my search for the most lightweight, flexible, powerful yet simple cryptographic Python library, I came across close to a dozen options.  After reading their documentation (cough!) and interfacing with their API's, I decided on python-crypto. The library has a wide collection of cryptographic algorithms and protocols, seemed like the best fit for what I was looking for, and is packaged by the major distributions. apt-get install python-crypto Next up, choosing the cipher Without going into detail, there are block ciphers and stream ciphers. They differ in how large a chunk of plaintext is processed in each encryption operation. A block cipher operates on a fixed-length group of bits (or blocks), for example 128-bits. In contrast, a stream cipher operates on relatively small blocks, typically single bits or bytes, and the encoding of each block depends on previous blocks. I chose to use a block cipher, in particular the Advanced Encryption Standard (AES), which has been adopted by the US government.   Finally, some example code from Crypto.Cipher import AES def encrypt(plaintext, secret) encobj = AES.new(secret, AES.MODE_CFB) ciphertext = encobj.encrypt(plaintext) return ciphertext def decrypt(ciphertext, secret) encobj = AES.new(secret, AES.MODE_CFB) plaintext = encobj.decrypt(ciphertext) return plaintext Because the plaintext needs to be a multiple of block size, we specify Cipher FeedBack (CFB) mode so we don't need to deal with padding. But, the secret too needs to be a valid block size (e.g., 16, 24, 32), so for ease of use, we can use a lazysecret to achieve this. def _lazysecret(secret, blocksize=32, padding='}'): if not len(secret) in (16, 24, 32): return secret + (blocksize - len(secret)) * padding return secret CRC (Cyclic redundancy check) As mentioned above, we need to verify data integrity, as well as perform a check that decryption with the provided secret key was successful. During encryption plaintext += struct.pack("i", zlib.crc32(plaintext)) During decryption crc, plaintext = (plaintext[-4:], plaintext[:-4]) if not crc == struct.pack("i", zlib.crc32(plaintext)): raise CheckSumError("checksum mismatch") In the above example, we are adding a 4 byte CRC of the data, onto the data, prior to encryption. After decryption, the CRC is recalculated and matched against the attached CRC. If they match, all is good in the world. If they don't, well, not good... Update: Thanks to tptacek on Hacker News for pointing out that a CRC is a non-secure hash function designed to detect accidental changes, and should not be used as a security check. Instead, it's recommended to use a secure hashing function such as SHA1. Putting it all together crypto.py import zlib import struct from Crypto.Cipher import AES class CheckSumError(Exception): pass def _lazysecret(secret, blocksize=32, padding='}'): """pads secret if not legal AES block size (16, 24, 32)""" if not len(secret) in (16, 24, 32): return secret + (blocksize - len(secret)) * padding return secret def encrypt(plaintext, secret, lazy=True, checksum=True): """encrypt plaintext with secret plaintext - content to encrypt secret - secret to encrypt plaintext lazy - pad secret if less than legal blocksize (default: True) checksum - attach crc32 byte encoded (default: True) returns ciphertext """ secret = _lazysecret(secret) if lazy else secret encobj = AES.new(secret, AES.MODE_CFB) if checksum: plaintext += struct.pack("i", zlib.crc32(plaintext)) return encobj.encrypt(plaintext) def decrypt(ciphertext, secret, lazy=True, checksum=True): """decrypt ciphertext with secret ciphertext - encrypted content to decrypt secret - secret to decrypt ciphertext lazy - pad secret if less than legal blocksize (default: True) checksum - verify crc32 byte encoded checksum (default: True) returns plaintext """ secret = _lazysecret(secret) if lazy else secret encobj = AES.new(secret, AES.MODE_CFB) plaintext = encobj.decrypt(ciphertext) if checksum: crc, plaintext = (plaintext[-4:], plaintext[:-4]) if not crc == struct.pack("i", zlib.crc32(plaintext)): raise CheckSumError("checksum mismatch") return plaintext   Interfacing with the above code couldn't be simpler. >>> from crypto import encrypt, decrypt >>> ciphertext = encrypt("confidential data", "s3cr3t") >>> print ciphertext \hzÛÍúXM~âD÷ð¿Bm >>> print decrypt(ciphertext, "s3cr3t") confidential data >>> print decrypt(ciphertext, "foobar") Traceback ... <class 'crypto.CheckSumError'>   Ever needed to use encryption in a Python project? Leave a comment! [Less]

9 things I didn't know about CSS that I should have known years ago but I only picked up from a couple of good CSS books. Having dabbled in web development over the years, I've had a basic working knowledge of CSS for quite some time. In fact, I've ... [More] been around long enough to remember what the web was like when we were all using font tags and transparent pixel tricks. Back when it wasn't unusual to come across a web site using the BLINK tag. Let me tell you it was ugly and it was a mess. (Now get off my lawn!). As soon as browsers started supporting CSS I picked up just enough to save myself from committing the geek equivalent of a war crime, but I never got around to fully mastering CSS in all its glory. Advanced subjects such as floats and layout still seemed a bit mysterious. And every so often I would come across a bit of CSS code that made no sense yet rendered perfectly in every single browser I tested. Clearly my CSS skills were behind the times. <!--break--> Then a few months back I started working on a few improvements to the TurnKey web site and decided it was time to get back to basics. So I bought a couple of books which were recommended by a friend: CSS: The Missing Manual CSS: The Definitive Guide, 3rd edition If you need to pick just one I would say both were good but I think the definitive guide has an edge. It's better organized, more comprehensive, and a better reference. It also provides better "theory" that helps create a useful mental model of how things are supposed to work. OK, now on to the list: Learn how to use FireBug to test CSS on the fly: in hindsight, the most important thing I learned while reading these books and experimenting wasn't directly related to CSS itself - it was how to efficiently use FireBug to test CSS code on the fly. Previously I would write my CSS in a file text editor and then tweak the values in FireBug's inspection mode. I found myself hitting reload a lot and there was always a lag between writing CSS and seeing its effect on the web page. I found non-trivial tweaks to the CSS were much easier to write/experiment with in FireBug's "edit" mode, which just lets you mess around with the CSS in a simple editor and immediately applies those changes to the currently displayed browser window. In this mode you have to watch out not to hit reload or you will loose your CSS edits. After things are done, you can copy the result to a permanent file containing the stylesheet. Modern browsers (gasp) follow the rules: with regards to CSS itself the main thing I learned is that there are precise, non-browser specific rules governing web site presentation and lay out. Those of us who learned the ropes in earlier times when older browsers such as IE6 ruled have this misconception that browsers may interpret your CSS unpredictably. Modern browsers can be relied on to lay out your web site according to a rigorously defined, standard box model with rules that aren't so difficult to understand. What this means is that nearly all of the things I tried in Firefox 3.5 looked pretty much the same in IE7 and IE8. Down to the pixel level. And you may need that sort of pixel level control to implement visual effects such as rounded corners. Use the background Luke: the visual workhorse in CSS is the background family of properties: background-color # `transparent' is a color background-repeat # repeat-x repeat-y no-repeat background-image background-position "background" is a convenient short hand when you don't want to define background properties separately: background: <background-color> <background-image> <background-position> The primary limitation with the background property is that an element can only have one background. In some cases when that isn't enough you'll see designers inserting empty <div> hooks which are precisely positioned to create the illusion that a graphical element has multiple background (e.g., the round corners of fluid / variable-width element) Advanced selectors: CSS allows you more flexibility in selection than just the "standard" ancestor / descendant relationship: body h1 { # this matches any h1 element inside any other element inside # the body } You can also select direct children: body > h1 { # only h1 that is the direct child of body will match } Siblings: h1 + h2 { # only h2 that comes immediately after h1 will match } The first child of an element: ul li:first-child { # only matches the first li item } Global selector: * { # matches any element } *.myclass { # the better known .myclass is just shorthand } *#myid { # the better known #myid is just shorthand } Attribute selectors: img[title] select only img tags with "title" attribute input[type="text"] input fields of type text a[href][title] select a that has both href and title attrs *[title] any element that has title attribute img[title~="Figure"] any img with title that contains Figure in it regexp match? img[title^="bar"] any img with title that starts with "bar" img[title$="bar"] any img with title that ends with "bar" img[title*="bar"] any img with title that contains substring "bar" Things to remember about absolutely positioning elements They are taken completely out of the presentation flow. No space is made on the page for that element. They are positioned relative to the nearest absolutely or relatively positioned ancestor element (or otherwise the body). In other words, if you want to position element A relative to element B, you can put element A inside element B and define B as relatively positioned. You don't have to actually define an offset for the relative position of B. Margins collapse: margins are not just padding that goes beyond an element's borders. Margins collapse so that if the margins of two elements touch the smaller margin will collapse. They won't be accumulated! In other words if M1 > M2, M1 + M2 = M1 Any element can be turned into an inline or block element: display: inline|block Block elements and inline are elements are different in that blocks will insert a break and expand to the available in their parent box UNLESS THEY ARE FLOATED. By contrast, inline elements are happy to live side-by-side with other inline elements. The rules for floating is precisely defined by the standards. That's important because you should be able to rely on them down to the pixel to figure out in advance if an element will be floated along side an existing element or below it. You can forget about older broken browsers such as IE6 where you would sometimes get a broken result. The precise rules are a bit complex, but you don't really need to know them unless you need pixel perfect perfection. A few tips to help you get by: Floated elements are rectangle block boxes who's border dimensions are calculated to exactly fit their contents + padding + border. In other words, a float is as small as it has to be. Floats can float up and right but never up. So they'll never show up higher in the page than where they are inserted. In Firefox (tested up to 3.5) a parent element that contains floated elements doesn't expand to contain its floated children unless it's either also floated or has the "overflow: hidden" property. This confused me a bit as I didn't expect floated elements to spill out of the bottom of the containing element. Relative/absolute positioning is also rigorously defined. Once you know the rules you should be able to get predictable results. If you experiment in a modern browser you can expect the result to look the same in other modern browsers. Coming up next week: my CSS cheat sheet. [Less]

So you're building a web application, and using the excellent contrib.auth subsystem to manage user accounts. Most probably you need to store additional information about your users, but how? Django profiles to the rescue! Django provides a ... [More] lightweight way of defining a profile object linked to a given user. The profile object can differ from project to project, and it can even handle different profiles for different sites served from the same database. In a nutshell, using Django profiles consists of 3 steps: Define a model that holds the profile information. Tell Django where to look for the profile object. Create the user profile as needed.   Defining the user profile model The only requirement Django places on this model is that it have a unique ForeignKey to the User model, and is called user. Other than that, you can define any other fields you like. For our example, we'll create 2 user profile fields (url and company). account/models.py from django.db import models from django.contrib.auth.models import User class UserProfile(models.Model): user = models.ForeignKey(User, unique=True) url = models.URLField("Website", blank=True) company = models.CharField(max_length=50, blank=True)   Tell Django about the profile object Accessing a users profile is done by calling user.get_profile(), but in order to use this function, Django needs to know where to look for the profile object. This is defined in the projects settings.py file. AUTH_PROFILE_MODULE = "account.UserProfile" Important note: The settings value is appname.modelname, and not appname.models.modelname as you might expect. The reason for this is that Django is not doing a direct import, but using an internal model-loading function. Once defined, and if the profile exists, accessing a users profile is simple. foo/views.py @login_required def view_foo(request): profile = request.user.get_profile() url = user_profile.url #OR url = request.user.get_profile().url   Create the user profile as needed Notice in the section above I mentioned "if the profile exists". This is because the get_profile() function will raise a DoesNotExist exception if the profile does not exist. One of the common solutions to this issue is to create the profile when a user is registered using a signal (see my post on Django Signals). Another option is catch the exception, and redirect the user to a profile creation form. You usually want to do this if the profile fields are mandatory. My personal favorite though, is to have the profile created automatically when referenced, with the added bonus of being able to reference a users profile as user.profile instead of user.get_profile() account/models.py ... class UserProfile(models.Model): user = models.ForeignKey(User, unique=True) ... User.profile = property(lambda u: UserProfile.objects.get_or_create(user=u)[0])   To reiterate, including the above enables us to reference the users profile with cleaner code, and have it created if it does not exist. foo/views.py @login_required def view_foo(request): url = request.user.profile.url   Ever needed to store additional user information? Post a comment! [Less]

Ever tried logging into a machine with ssh and found you have to wait much longer than reasonable for the session to start? This happened to me a few times and was especially annoying with machines on my local network (or a VM attached to a virtual ... [More] network) that should be letting me in immediately. I eventually got mad enough to strace the SSH daemon and debug what was going on and it turns out it's a DNS thing. Basically the session is slow to start because the SSH server is trying to lookup the hostname of the SSH client and for whatever reason it's timing out (e.g., it can't reach a nameserver, because you happen to be offline) There are a couple of very simple ways to fix that: add "UseDNS no" to /etc/ssh/sshd_config add the client's net address to the server's /etc/hosts Was that helpful? Has this ever happened to you? Post a comment! [Less]