Do It With Drupal in review

Matthew Nuzum —  — 7 Comments

*Please let me know if you find errors in this document or if you’d like me to add clarification. These notes are based on my hearing of the conference sessions and it’s entirely possible I mis-understood the speakers or in the heat of the moment a slight error was communicated.*

Drupal is the software we use to manage our main websites. Last week while UDS was happening I was able to attend a training seminar in New Orleans, Louisiana called [Do it with Drupal](http://www.doitwithdrupal.com/) to learn how to better make use of this platform. It was a powerful training program and I came away with a wealth of information that will help us in the future.

This report is quite long because I know that a number of people will be interested in the various parts, so I’ll start with a one-page summary:

## My Goals for This Seminar: ##

1. Make it easier to delegate tasks to others
2. Simplify common tasks for users creating content
3. Evaluate the process of upgrading to Drupal 6
4. Learn about Acquia, a professionally supported version of Drupal created by the Drupal founder and various other core developers

## Nutshell: ##

* Drupal 6 is a major improvement, in large part because a significant amount of time was spent re-architecting key add-on modules such as views and CCK.
* Future Drupal upgrades will not be as challenging because the key modules will not go through this same upheaval in the foreseeable future (I go this from the mouths of the CCK, Views and other modules developers)
* My observation is that Drupal developers are focused on the needs of integrators more than end users – this makes it a powerful tool with a wealth of APIs and add-ons, but it is still a challenge to use it as an end user CMS
* There are some major sites doing some cool stuff with Drupal – Sony/BMG, Warner Brothers and a few newspapers were demoed
* We at Canonical are using only a tiny subset of the most basic features – I can see some great benefits for us if we adopt some of the more advanced Drupal functionality. Potentially this could allow us to have fewer websites, manage content more efficiently and do some things we can’t do now.

(by the way, this last week the BIS team at Canonical released an open source plugin that allows drupal websites to [authenticate and get group information from Launchpad](http://news.launchpad.net/cool-new-stuff/new-launchpad-plugins-for-drupal))

## Key Modules: ##

Some of the powerful applications demoed relied on a few key modules, namely:

* Views 2
* CCK (aka Content Construction Kit)
* Flag
* Advanced Help
* Image Cache (powerful image uploading/resizing)
* Numerous CCK related fields
* Panels (Currently still relies on Drupal 5, same author as Views and Views2, he is now focusing his efforts on updating this powerful module)

Other interesting modules were:

* Organic Groups
* Workflow
* Revision Moderation (combined with workflow eliminates the need for a staging server)
* Date and Calendar (to replace event)
* Admin Menus (when logged in gives quick access to admin function)
* Markdown (wiki syntax for drupal)
* Developer Module
* Boost – big performance boost for sites with mostly anonymous traffic

Also, someone said I should check out “Selenium macro recorder” to record web-browser interaction and then re-play it later. Sounds slick.

/end one page summary

## Session breakdown: ##

## Keynote by lullabot, seminar organizers: ##

Drupal is not just a CMS, it is a framework for building web applications (as discovered later in the program and outlined below many of these can be built with no code).

Drupal’s flexibility comes from its lack of assumptions, which is makes it less usable.

The last 10% of a project (the polish) often takes 90% of your time – not a surprise to many of us.

Use the code “DRUP4” on O’reilly website to get a big discount on the new Drupal book soon to be released (I believe called “Using Drupal” co-authored by Angela Byron – supposedly teaches by walking through building several drupal websites).

Good explanation of open source philosophy including referencing how Sony/BMG sponsors a lot of Drupal development which often directly benefits Warner Brothers, their major competitor, who then feeds testing and bug reports back. (don’t tell the PHB’s)

## How we built Twitter with Drupal (Drupal fantasy site) ##
[http://www.doitwithdrupal.com/sessions/how-we-built-twitter-drupal](http://www.doitwithdrupal.com/sessions/how-we-built-twitter-drupal)

Goal, build a twitter clone with Drupal

* Don’t really make twitter with drupal
* It’s like blogging but you only post the title
* Ruby on Rails is lighter weight than drupal (but D has more features)
* Completed in 24 hours
* Views 2 module with 5 views total
* Flag module for following users and favorite posts
* About 200 lines of PHP needed, mostly to tie loose pieces together including the user information block
* Page title module
* Only one content type – custom – only contains body
* Custom page.tpl.php
* Form api gives validation for free
* Drupal builds up info and then pushes it through the theme which only shows the parts it wants (contrast to other systems where the theme calls functions to build the dataset)
* Templates – page, page-node, block, node, views-table, views-fields
* “posted x min ago” done with “Format interval function” “time ago” in date format of views

## Acquia info session ##

Acquia does for Drupal what Canonical does for Debian / Ubuntu – creates a commercially supported version and gives it away for free but adds value (including a nifty administration interface) for those who want to pay for commercial support.

Started by Dries Buytaert, creator of Drupal, accompanied by several core drupal contributors.

Includes Mollom, advanced spam blocking solution that passes good comments and prompts with a captcha for suspicious comments. Offers hosted search that is better integrated into the site’s unique features than a custom google search box but much more powerful than Drupal’s built in search.

Web, phone or email support available depending on pricing tiers.

[http://acquia.com/](http://acquia.com/)

## Flickr with Drupal (Drupal fantasy site) ##
[http://www.doitwithdrupal.com/sessions/how-we-built-flickr-drupal](http://www.doitwithdrupal.com/sessions/how-we-built-flickr-drupal)

Goal, clone Flickr using Drupal 5

* Modules: Views, Panels, Image Cache, Swif uploader (for multi upload)
* Panels of views
* Blocks and panels can be combined
* Blocks are best when they will be on multiple pages across the site
* Blocks can only be positioned in one place (you can’t say, “on this page put it here, on that page put it there)
* Panels can put blocks anywhere and can re-use any existing block
* Panels can pass special arguments into views and can accept args
* Views for gallery listing, don’t use image module, use Image field (for CCK) instead – image node is rough

## Basecamp built with Drupal (Drupal fanatasy site) ##
[http://www.doitwithdrupal.com/sessions/basecamp-built-drupal](http://www.doitwithdrupal.com/sessions/basecamp-built-drupal)

Goal: Build a basecamp clone in drupal
**note: This was one of my fav sessions. Webchic is an excellent instructor plus the end result looked like something truly beneficial and not just a show-off site.

Features:

* Write board is kind of like a wiki / blueprint
* Milestones are calendars / events
* Todo list
* Messages
* File uploads
* Chat (not implemented, felt it was superflous)
* People and roles/permissions
* Projects

Uses:

* Organic groups (for projects)
* OG Access control ?? for project privacy
* File uploads = CCK + file field
* Todo lists = CCK + flag
* Wikis = CCK + Markdown
* Calendar = Date + Calendar

* File field has advantages over Upload (Upload is scorned pretty much across the board during the seminar for being messy and simplistic), configurable by filetype
* Flag is awesome – it provides a toggle on/off for everything
* Date is better than event (also scorned severely)
* Zen is a good base theme that can be styled through CSS alone (no PHP needed)
* Subscriptions & Notifications framework
* CCK lets you have node references like foreign keys
* Core profile module allows you to add fields to users
* In a view an “attachment” is a sub-view
* The API hook “form alter” lets you modify forms

## Get satisfaction keynote by Lane Becker ##
(regarding community building, one of the tracks for the seminar)

* People are fun, radical and interesting. Building tools for them is fun, radical and interesting
* “It’s broken until it’s done.”
* The role “Community manager” is very broad and not very specific, kind of like webmaster
* When starting his company he picked a picture from flickr (creative commons license) that he felt identified the personality of his company
* Twitter’s imagery (and corporate identity) is built in and evident through and through
* “Myspace is so far beyond sell-out. I guess when you start at sell-out and keep going where do you end up?” (regarding that they have ads for their background images)
* “You’re all jerks” (to the audience, then a minute later) “Number one rule, don’t get up on stage and call your audience jerks. What was I thinking?”
* The difference between a church and a stadium is the design – it communicates to people what the expectations are
* Dogster founder realized in week 3 that he wasn’t in charge of his site anymore (the community was)
* BIG GOAL: What is your purpose, what kind of community are you building?
* Who belongs? Persona – describe the ideal community member
* Who doesn’t? Who are you trying to keep out?
* Building a leader board says competitive (cf: Digg)
* Outline your expectations
* What does success look like?
* Look for metrics that will accurately measure success

## Content Construction Kit (aka CCK) by Karen Stevenson ##
[http://www.doitwithdrupal.com/sessions/content-construction-kit](http://www.doitwithdrupal.com/sessions/content-construction-kit)
Karen is the author of CCK

* Use CCK instead of views when the extra bit of data is integral to the data, for example the “Artist” field for an album.
* Strongly suggest “advanced help” module, CCK uses it extensively
* Only use shared fields selectively – Use new fields unless necessary
* For example, a phone number field shared between “company” content types and “person” content types allows you to create a view of phone numbers
* Shared fields let you set up views that have an exposed filter that can show multiple content types
* Node reference is very powerful
* Advanced help for CCK includes docs for themeing (I looked at this on the airplane home – it indeed is very powerful and helpful)
* Contemplate (??) module (??) is a template editor
* Templates are used to theme CCK content (see advanced help for suggested file names for your templates which allow you to have great control over the look of your content. Once demoed you’ll see that this is very very powerful at creating custom web applications with no code.
* Diff module integrates nicely with CCK so you can compare one revision to another
* devel_generate (??) is a module which fills your site with sample content (useful for themeing)
* Computed fields are very powerful
* Leave CCK permissions off until your site is created or you may not see the fields you create – fields without permissions cannot be viewed – i.e. default to hidden
* Content types can be exported and imported
* “Node save” to create nodes automatically (??)

## Basic Views (Views2) by Earl Miles ##
[http://www.doitwithdrupal.com/sessions/basic-views](http://www.doitwithdrupal.com/sessions/basic-views)
Earl is the author of Views and Panels

* Views don’t need to be in a database, they can be created in a module
* Sidenote: Many bigger sites (ubuntu.com included) have a module of site-specific functions – you can put your views into these modules and track them with version control
* Overridden views are code based views that have been modified for the local site
* There are three questions:
1. What to display? (also include fields for sorting and filtering even if hidden)
2. How do you want to display it? (default is unformatted)
3. Additional display options: feed, page, block, attachment
* When looking at fields on the Views page italicized means default value, bold means overriden
* IMPORTANT: You haven’t changed anything until you hit “SAVE” – refresh if you leave the page and come back and you think you’ve lost your data since the values are stored in your session before saving
* View name must be machine readable – description should be human readable
* You can export your views and put them into code (aka your site’s custom module)
* Views is just an SQL query builder and shows you the SQL at the bottom of the page so you can modify it or optimize it when putting the view into a module
* Data does not have to be in the same database as your drupal site – it can be in any database the drupal site has access to but it must be in the same type of database (i.e. MySQL sites can use any MySQL database) – this could allow one site to syndicate content from another
* Modules to investigate: Node que, views bulk operations, views bonus pack, views data source (jsonp, xml)

## Organic groups ##
Looks powerful but I skipped this because it looked like something I won’t need right away. There will be a video I’ll watch down the road.

## Jquery by John Resig ##
[http://www.doitwithdrupal.com/sessions/powerful-interfaces-jquery](http://www.doitwithdrupal.com/sessions/powerful-interfaces-jquery)
John is the author of jQuery

Session was mostly the absolute basics, though I did learn stuff about chaining and some shortcut functions.
#(‘form’).submit() adds a callback to the form (for validation, etc), it does not submit the form.
#(‘#somediv’).load(‘file.html’) grabs the body tag of the file file.html and inserts it into #somediv
#(‘#somediv’).load(‘file.html h2’) grabs the h2’s from file.html and inserts them into #somediv

Check out “theme roller” for jquery ui – gives you a gui to customize the themes for the jquery ui components

To clarify the meaning of “this”:

$(‘a’).each(function() {
this = ‘
$(this) = [
] (a jquery set containg the current tag)
$(‘this’) = [] (an empty set because there’s no )
});

Talked with John outside the session, he’s working on “Sizzle” which is a high performance and library independent selector library. All the tools (jquery, dojo, yui, etc) have basically standardized on a selector syntax but each have their own implementation. Sizzle will be able to plug into any of the tools and many of the major libraries are planning to integrate it (dojo, yui, jquery all mentioned I believe).

## Themeing by (I won’t say who) ##

What a train wreck. I learned little. Poorly organized, poorly planned, poorly presented. Here are my notes supplemented with some info I picked up elsewhere:

* Fine-tune your data’s appearance and presentation in properly named .tpl files in your theme folder. For example, node-story.tpl.php. Views’ and CCK’s advanced help menus will tell you what to name your tpl files and even give you boiler plate code to paste in and customize.
* There’s a module called devel module that gives you a firebug-like interface where you can click a section of a page and it will tell you what to name your .tpl files to override and what functions where used to produce the output.
* In Drupal 6 there are pre-process functions where are supposedly quite powerful
* For a bare-bones theme check out hunchbaque

## Multimedia Handling by Nate Haug ##
[http://www.doitwithdrupal.com/sessions/media-handling](http://www.doitwithdrupal.com/sessions/media-handling)
Nate was another awesome presenter, however this topics scope was quite broad so we didn’t get to cover nearly enough info.

Tail end of the session was a presentation by Kaltura which demoed an awesome web-based tool that allows you to create your own video mashups from stock media, media you upload and even directly from a web-cam connected to your PC: [http://drupal.kaltura.org/](http://drupal.kaltura.org/)

* Image / Video / Audio modules are pretty much DEAD
* Use CCK / Views2
* Drupal 6 has improved things a lot
* Don’t use the upload module (he said this quite emphatically)
* Image galleries can be easily created with CCK / Views / Image field / File field / Image cache / Image API / Actions
* Image cache was especially interesting. I was concerned about performance, talked to the guys doing the performance session who assured me that it was well thought out and implemented so that heavy caching was done
* Image cache module basically resizes or adjusts images on the fly in the browser. Quite powerful and has add ons for doing a multitude of adjustments beyond resizing, cropping and titling that is built in.
* Audio – CCK / Views / File field / file field podcaster / SWF tools / 1 pixel player
* Video (self hosted) – CCK / Views / File field / SWF tools / JW Player or Flow Player
* Video (offsite / youtube) – CCK / Views / Embedded media field / SWF tools / JW Player / Flow player
* Transliteration module very useful for generating url friend file names
* Image resize filter – resize images for IMCE (a multi-purpose uploader that does not use CCK)

Things to look into: What is Red 5 (something to do with recording video using flash and a webcam)?

## Advanced Views by Earl Miles ##
[http://www.doitwithdrupal.com/sessions/advanced-views](http://www.doitwithdrupal.com/sessions/advanced-views)
Earl is the author of Views and Panels

* Related fields – foreign keys
* I asked if you could add values to the related fields list while creating or editing content – no, not at the moment
* Arguments can be passed into views, for example from the URL or logged in user id, used for filtering etc – cannot be used by blocks
* Validator can check args before passing into views
* Summary view – for views when an argument is left out – for example if you have a view that shows up at somesite/events/2009-01-01 then a summary view can be shown at somesite/events/
* Themeing – see advanced help for template (.tpl) file names for fields or views – more specific names will override templates with less specific file names
* CCK fields, especially multiple or shared fields impact performance
* You can add indexes to the database tables to improve performance (CCK fields don’t get indexes by default I believe) but be careful, sharing a field changes the DB structure so you may need to recreate your indexes when sharing a field
* Views shows the SQL – use explain to learn how to optimize
* Attachments allow you to combine multiple views on one page
* Views calc (??) is a module that allows you to do summing / totaling at the bottom of a view (imagine a table of values)

## Dates and Calendar by Karen Stevenson ##
[http://www.doitwithdrupal.com/sessions/date-and-calendar](http://www.doitwithdrupal.com/sessions/date-and-calendar)
Karen is the author of CCK and the Date and Calendar modules

* Date widgets (of CCK) can be in different formats, Karen prefers ISO format (aka 2008-12-15), can be input as:
* text field
* select list
* calendar pop-up
* Use the calendar module
* Recurring dates is in transistion, you can use:
* repeating date field
* a date field with multiple dates
* date field with multiple values (this wasn’t clear to me but hopefully some experimentation will enlighten me)
* Export ical is done and working nicely
* Import ical is hard but improving steadily

(Sorry, at this point of the conference my brain was getting tired)

## Performance and Scalability by Matt Westgate and John VanDyk ##
[http://www.doitwithdrupal.com/sessions/performance-and-scalability](http://www.doitwithdrupal.com/sessions/performance-and-scalability)

* Use mod_expires (for apache)
* CDN can help
* Boost module for drupal serves content as static for sites with mostly anonymous users
* Opcode caching like APC or xcode
* Query caching (a MySQL setting) can have a big impact
* mem cache is a BIG WIN – look for the memcache module
* puts sessions table into memcache
* puts url alias table into memcache
(from my experience, using mysql and default myisam table there can be a lot of contention for the session table – I can see this being a big win)

**Apache**

* Properly adjust maxclients (simplistic formula, see how much RAM an apache child uses then take total ram (not counting swap) you can allocate for apache and divid by the amount used per apache child and use this value for maxclients – setting too high hurts performance as your site gets busy
* Keep alive on, time out 2

**PHP**

* Opcode cache APC / Eaccelerator / xcode
* max_execution_time 60
* max_input_time = 60
* memory_limit = 75 – image cache benefits from extra ram

**Drupal**

* More modules make the site slower
* When writing code, unset() on large views frees up memory

**MySQL**

* in ini file, [mysqld] query_cache_size = 32m
* [drupal] Views are SQL queries, use explain to help optimize them
* Log slow queries and hand optimize them

**Cron**

* Keeps the database clean

**Caching tweaks**

* Aggressive cache (setting in drupal)
* min cache lifetime
* CSS preprocess (there’s something called aggregator that combines css files and also javascript files for fewer http requests)
* A JS/CSS compressor is in works for D7

**For Devs**

* use cache_get() and cache_set() for storing values in a cachable way

If using NFS (and many large sites do use it) make sure the PHP upload dir is also on NFS

## Final Notes and Thoughts: ##

* When upgrading from D5 to D6 read and follow the instructions, especially for CCK.
* There’s no upgrade path for views/image field.
* You can use the combination of Workflow and Revision Moderation modules to remove the need for a staging server. Instead, create content on the live server and have a manual review process to publish changes (even changes to existing pages) live when ready. Assign the ability to publish to different roles if so desired.
* Blocks are not versioned, someone suggested using views to create blocks that are versioned (research needed)
* Remarkable by its absence was any kind of discussion on security
* Based on the discussions, end-user usability is not high on the priority list of developers. The target audience is technical integrators and people who are willing to RTFM. The only person who mentioned usability in the sessions I was in did so while making an erroneous (or at least contestable) statement about usability – specifically that users find a single check box more usable than radio buttons offering the option of yes / no for a boolean value.
* When asked during the wrap up multiple panel experts laughed at and mocked the idea of it being a good thing to offer users a WYSIWYG editor. This is on the podcast at [http://www.lullabot.com/audiocast/podcast-67-diwd-08-live](http://www.lullabot.com/audiocast/podcast-67-diwd-08-live) (if you care to listen to a 57m podcast). A panel expert literally said that we should teach our users HTML if they want to edit content on the site. I was very disappointed at this statement – I spend a significant amount of time helping users and it would be nice if I could spend less time helping them and more time doing the stuff that can’t easily be delegated. Despite the lack of interest by those at the conference there are good wysiwyg options available and their support is being improved on via the “wysiwyg module”.
* I feel I accomplished all my goals though the first two listed were realized better via my conversations with other users rather than developers. Jeff Eaton of Lullabot was especially helpful at discussing technical topics relating to deploying content across sites and Mark Jarrell of Jones Knowledge Group demonstrated what he’s doing to solve this problem (workflow + revision moderation modules).

Thanks everyone!

Matthew Nuzum

Posts Twitter Facebook

Web guy, big thinker, loves to talk and write. Front end web, mobile and UX developer for John Deere ISG. My projects: @dsmwebgeeks @tekrs @squaretap ✝
  • Wow! What a great writeup, Matt! Looks like I don’t have to do one for myself; I’ll just link here. 😉

    My favorite presentations were probably Lane Becker and building Basecamp. They generated the most “ah ha” moments for me. I didn’t get as much out of the module-specific presentations because I’ve already spent a fair amount of time with them already. All good stuff though.

    You had a few ??’s in the CCK summary. I’ll just confirm what you probably already know. The automatic content generation is part of the Devel Module. I think it is a MUST HAVE… especially for themeing. The Contemplate Module is neat for allowing you to change how things display in a point-n-click way, but I’m not sure how it would fit in a broader workflow. And finally, I’m not sure what “node save” is.

    BTW, it was great to meet up with you again. We should get together for lunch sometime.