Cerb

Release notes for Cerb 6.7 Cerb (6.7) is a major functionality release candidate as of April 8, 2014. It contains over 75 new features and usability tweaks from community feedback. See: http://wiki.cerbweb.com/6.7 [Workspaces/Dashboards] ... [More] Dashboards can now be configured to display widgets in one, two, three, or four columns. Extra columns are added to the right without rearranging existing widgets. When removing columns from a dashboard, widgets are automatically and evenly redistributed, and can be dragged into a desired order from there. By using one or two columns, more space is available for detailed charts. Four column dashboards can display many compact widgets on a single page without scrolling, and make better use of larger displays. [Workspaces/Dashboards/Performance] The content on dashboard widgets is now cached for 60 seconds, which can dramatically improve performance with many concurrent workers. Manually clicking the reload button on a widget will update it with the latest data. When a dashboard is first displayed, any cached content is displayed immediately when the tab first draws. Any widgets with an expired cache are then displayed with a "spinner" image and fetched in sequence through an Ajax pipeline. This process is much more responsive than the previous behavior of waiting until all the widgets have loaded before displaying anything (and leaving workers staring at a mostly blank screen wondering if something is broken). It's also more memory efficient. [Workspaces/Dashboards/Usability] When a workspace dashboard tab is blank, a helper message is now displayed with instructions on how to modify the number of columns and add new widgets. [Workspaces/Usability] When a new tab is added to a workspace it is now automatically selected. Previously the new tab was highlighted but it still had to be clicked, which was only helpful in a situation where a worker intended to add several tabs at once before modifying any of them. This made the most common use case slightly more difficult. [CHD-3065] [Workspaces/Worklists/Usability] Worklists are now loaded in order on workspace tabs using an efficient Ajax queue. Previously this process attempted to only load worklists that were visible in the scrolled browser window, but this is no longer necessary with the workspace improvements in 6.x. The loading worklists have a more modern style and an animated "spinner" image. [Workspaces/Dashboards/Widgets/Performance] The caching on dashboards can now be configured per-widget. By setting a higher cache TTL (time-to-live) on infrequently used or changed data, page generation performance is improved. The default cache lifetime is set to 60 seconds on every widget during the migration. Refreshing a widget manually automatically uses the latest data and updates the cache. [CHD-3172] [Virtual Attendants/Mail] The 'initial message' and 'latest message' placeholders are now available in all message-based Virtual Attendant behaviors. [Virtual Attendants/Simulator/Usability] The Virtual Attendant simulator now only makes conditions editable if they are used by the behavior being tested. For instance, an auto-reply behavior that uses the conditions 'message is new' and 'message is outgoing' will now only show those two values in the simulator, which drastically improves usability. Previously, the values of every possible condition were shown (potentially hundreds) even though most of them weren't useful for testing the current behavior. [Platform/Dependencies/Twig] Updated the Twig template engine from 1.8.2 (2012-05-29) to 1.15.1 (2014-02-13). [Search/Performance] Full-text search results will now be retrieved as matching IDs before any other filters are applied. In most cases, this is much faster and more efficient than trying to do everything at once. This improvement makes it possible to consistently cache the results of full-text searches, even when other filters on the same worklist change. Previously, each variation of filters would require a different cache, and this could quickly overwhelm the default query cache settings in MySQL. Additionally, full-text results are now sorted by relevancy (best fit for the given search terms) and the top 250 matches are returned. Previously, relevancy wasn't considered at all, and any number of results were returned (even hundreds of thousands), which was very inefficient. The scope can be narrowed by searching for more specific terms. This "pre-fetch" step for full-text searching also makes it very easy for Cerb to integrate with other search engines like Sphinx, Solr, Elasticsearch, etc. [Search/Platform/Plugins] Cerb's built-in MySQL full-text search engine now supports attributes (textual or numeric). This makes it possible to segment the indexed content. For instance, the comment content index now has a 'context' attribute, which can restrict text matches to only comments of a specific type (ticket, task, opp, etc). This was a necessary change due to the other improvements in full-text searching, since it would have been possible for search text to match 250 comment records without any of them belonging to the appropriate type of record. Each search schema can define its own attributes, and these will be automatically added when a new fulltext_* table is created. Most specialized search engines (like Sphinx) already have the capability to track attributes, and this change will make it easier for Cerb to seamlessly integrate with them. [Placeholders/Snippets/Virtual Attendants] Added a 'secs_pretty' filter to placeholders. This displays a number of elapsed seconds as a human readable label like '2 hours, 5 mins'. [Search/Platform/Devblocks] New search engine platforms can be added to Devblocks as extensions. The default implementation remains MySQL Fulltext, but plugins can implement native support for Apache Solr, Sphinx, Elasticsearch, Cloudsearch, etc. [Search/Quick/Usability] Search engines can now provide their own quick search examples on a per-field basis. These are shown when a worker selects a fulltext field from the quick search dropdown. For instance, MySQL Fulltext and Sphinx have different search syntaxes. If a search index is moved to Sphinx, workers will be given the appropriate search tips. The quick search examples can also include custom examples per-index, so any additional fields indexed in Sphinx can be provided as examples (e.g. "@subject search text"). Any kind of content can be indexed as a field like this in Sphinx, and Cerb doesn't need to know about it for the queries to work. [Setup/Search] Search functionality can now be configured from Setup->Configure->Search. Indexes are shown for any records that have full-text searchable content, including those contributed by plugins (e.g. message content, kb articles, comment content). A search engine can be configured for each index individually. By default, all content is indexed in the built-in MySQL Fulltext search engine, and no special configurable is necessary. However, dedicated search engines like Sphinx offer more features and higher performance. [Search/Sphinx] Implemented support for the Sphinx search engine, an open source search server built for speed, scalability, and simple integration. This option requires Sphinx to be installed and configured separately, but you can expect searches to perform several times faster than MySQL Fulltext. Additionally, after switching to Sphinx you can drop the related fulltext_* tables in Cerb's database to reduce memory and storage usage, and to speed up processes like backups. Arbitrary fields can also be indexed by Sphinx and exposed to search (@status @subject @mask @headers, etc). This engine can be selected per-record from Setup->Configure->Search. Refer to the docs for setup instructions. [Platform] Cerb now requires the 'mysqli' PHP extension, since the 'mysql' extension is deprecated in PHP 5.5+ and will be removed shortly. [CHD-2954] [CHD-2549] [Performance/Scheduler/Maint] Improved the performance of the nightly maintenance scheduler job. Previously, this was inefficiently looking for broken links between records by performing expensive table joins. [Devblocks/Code Cleanup] Fixed an issue where negative byte values weren't formatted by DevblocksPlatform::strToPrettyBytes(). It's common to have a negative number of bytes when computing deltas between two numbers. [VAs/Placeholders/XML] Fixed the 'xml_decode()' function in placeholder scripts so it doesn't attempt to convert an XML document to an array. It now remains a SimpleXMLElement instance from PHP, which can be iterated, queried with XPath, etc. [VAs/Placeholders/XML] Fixed an issue with the 'xml_decode()' function in placeholder scripts where namespaced XML documents didn't always work with XPath queries. [VAs/Placeholders/XML] Added an 'xmlxpath()' function to placeholder scripts for querying the XML object returned by 'xmldecode()'. This can also return attributes. [CHD-2955] [Performance/Cache/Plugins] The full cache of extension properties was being cleared every time any property changed, which created some extra database traffic. This happened often during scheduler /cron jobs. Properties are now cached per-extension, so the invalidation of a cache has no impact on extensions that didn't change. [Cache/Performance/Platform] New cache engines can now be added through plugins. Caches improve the performance of Cerb by avoiding expensive database queries for content that hasn't been modified. Previously, only caching in the filesystem and memcache were supported. As before, objects are cached to the local filesystem by default without requiring any special configuration. Unlike before, the minimal set of 'bootstrap' caches required by the Devblocks platform are now always cached in the filesystem even when other cachers are available (e.g. plugins, extensions, global settings, tables, and classloader). This makes it possible to gracefully fail to disk-based caching if a previously configured plugin is no longer available, without introducing continuity issues. This also makes it possible to configure caching in a much more usable way from the browser in Setup, and to store cache configuration information in the platform. The cache-related settings in the framework.config.php file are no longer used and may be safely removed. If you were using memcache, you'll need to enable it again. [Performance/Cache/Platform] The platform-managed settings of each plugin are now cached individually. This allows a plugin to invalidate its cache without requiring the settings of all other plugins to be reloaded from the database. The settings for the 'devblocks.core' plugin are always cached in the local filesystem, so they're available as the platform is starting up. This is where settings are stored for cache engine extensions (which can't load their settings from themselves, before they are set up). [Cache/Performance/Platform] Server-side caching is now configured in Setup->Configure->Cache. Previously this had to be configured from the framework.config.php file, and only the filesystem and Memcached was supported. The cache engines provided by plugins show up here. When configuring a cache engine, its settings are tested before they are saved, and any configuration errors will be reported in the UI. [Cache/Platform] When content is cached by Cerb in the local filesystem in storage/tmp/, they are now named "cache--" rather than "devblocks_cache---". This is mainly aesthetic, but some cache engines do have a slightly higher memory cost for longer key names. Many filesystems have file name length limits around 255 characters. Additionally, the shorter prefix means that files can be distinguished from each other after fewer characters. [Cache/Security] When objects are cached to the filesystem, permissions no longer include 'world-readable' (in Unix parlance, they are chmod 0640). This helps prevent other users on the server from reading their content (Cerb still really shouldn't be installed in a shared hosting environment). [Platform/Plugins] It's very common for JSON objects to be saved as plugin settings. The 'DevblocksPlatform::setPluginSetting()' and 'DevblocksPlatform::getPluginSetting()' methods now support an additional argument for automatically encoding and decoding JSON-based settings. This saves some redundant code when those settings are loaded or used in multiple places. [Cache/Performance/Memcached] Memcached-based caching has been implemented as a plugin. It is configurable from Settings->Configure->Cache. The host, port, and key prefix can be set. A key prefix provides a way to namespace keys so that multiple applications can use the same Memcached datastore. A single host:port pair should be set, and a distributed setup should have multiple services placed behind a consistent hashing proxy like Twemproxy. This is a way more reliable way to provide highly available services, as opposed to handling them in the client (i.e. Cerb). [Cache/Performance/Redis] Redis-based caching has been implemented as a plugin. It is configurable from Settings->Configure->Cache. The host, port, auth, database number, and key prefix can be set. Auth is optional, and useful for publicly hosted Redis databases (like Redis-to-go); although ideally Redis should be configured on an internal IP and be placed behind a firewall. The database number should be left blank when using a proxy in front of Redis, as many don't support multiple databases (and always use number zero). A key prefix provides a way to namespace keys so that multiple applications can use the same Redis datastore. A single host:port pair should be set, and a distributed setup should have multiple services placed behind a consistent hashing proxy like Twemproxy. This is a way more reliable way to provide highly available services, as opposed to handling them in the client (i.e. Cerb). [Performance/Worklists] When a worklist has only one page of results with fewer rows than the paging limit (e.g. 6 rows out of 10), then an extra database query will no longer be executed to compute the total rows. The total records is simply the number of results on the first page. The extra query generally wasn't expensive in most cases, but it could potentially have run thousands of times per day on a moderately busy system. [Performance/Platform] Several of the original database tables (e.g. created in 4.x and early 5.x) had a redundant unique index for the 'id' column. This isn't necessary since those tables already have a unique primary key. This added a slight delay when modifying records, and it wasted a little disk space. These redundant indexes are cleaned up by the update. [Performance/Activity Log] The insertion of activity log entries has been optimized. Previously each insertion ran two database queries: one to insert a blank row and retrieve the new ID, and the second to update it. The same insert/update pattern is used elsewhere, but it's not necessary here because activity log records are never modified. This process now happens in a single query. [Performance/Watchers/VAs] The code that retrieves a list of watchers for a record has been optimized. Previously, this was inefficiently finding results with a JOIN between links and worker records, which always bypassed the worker cache. It could also run a second query to retrieve worker data in some situations. The links are now retrieved as IDs without a JOIN, and the worker cache is always used to return record data. [Performance/Mail/Parser] Lookups are now cached when checking if an email address is owned by a worker. This functionality is used frequently in the email parser where every bit of extra performance helps. [Performance/Mail/Parser] Mail routing rules are now cached when processing new email. This functionality is used frequently in the email parser where every bit of extra performance helps. The cache is automatically cleared every 20 minutes in order to rebalance the order that rules are checked in based on how often they match. [Performance/Mail/Parser] Multiple message headers are now inserted into the database in a single query. Previously, headers were inserted individually. Early profiling shows a 33% reduction in total queries when inserting many tickets at once (which the scheduler does quite often when polling POP3 mailboxes). [Performance/Mail/Anti-Spam] Optimized the method used to compute spam probabilities on incoming mail. Previously, ticket and message data was always retrieved from the database, and scores were always saved directly back to ticket records. Now arbitrary content can be analyzed and the spam score and interesting words are returned without extra database queries being performed. In the parser, the anti-spam fields can be set at the same time as everything else, which eliminates several redundant queries for every message being processed. [Mail/Parser/Anti-Spam] Previously, the anti-spam algorithm split subjects like 'CamelCapsWordsWithoutSpaces' into multiple words before analyzing them. Our research shows that these spam subjects haven't been popular for many years, so this code has been removed. Additionally, this approach had the side-effect of not analyzing words at all when they alternated cases every letter 'LiKe tHiS'. This content is now converted to lower-case and compared like normal. The subject of an email message generally only represents a small portion of its overall content, and this change shouldn't have a negative impact on spam classification. [Performance/Mail/Routing] The parser's mail routing behavior no longer directly modifies records in the database. Instead, changes are made to the in-memory 'model' that the parser manages before a ticket is created. Any custom field values that are set from routing are now handled by the same code that sets custom fields on mail filtering behaviors (combining both actions into a single operation). [Performance/Mail/Parser] Previously, the mail parser created a ticket record in several steps (ID, meta, message links, spam score, group and bucket). For instance, this prevented Virtual Attendants from detecting ticket changes too early in the process before the associated records were also inserted (custom fields, messages, activity log, etc). While this worked, it had the downside of being quite inefficient, because event listeners and Virtual Attendants still reacted to every record change, but just ignored those events when the fields they cared about weren't present. After a few design changes in the parser, tickets are now created in a single step, after all their related records have been created. This is far more efficient, and early profiling shows a decent performance gain. [Cache/Platform] The cache service can now be instructed to store certain keys in local memory for the duration of the current request. This is useful when the same underlying data is used over a short time period by multiple features without any communication between them (e.g. worklists, Virtual Attendant behaviors). This can also be used when the cached data changes to frequently to persist for a longer period of time, or when the active cache engine can't manage volatile objects. [Cache/Filesystem/Platform] The filesystem cache engine now supports exclusive locking when writing to files. Previously, in busy environments it was possible (albeit rare) for two webserver processes to simultaneously write to the same cache file, which could lead to inconsistent file contents. [Cache/Redis/Platform] The Redis cache engine now generates a unique key prefix every time the cache is globally cleared. This effectively namespaces keys so that old keys will become inactive and be automatically cleaned up (assuming Redis is running in 'maxmemory' mode). This is proxy- and multi-tenant friendly since the 'flush' commands are never used. Objects are now given a default time-to-live of 24 hours to help facilitate this. This approach enables a lot of dynamic content to be cached without concern over exhausting the available memory (e.g. searches, records, worklists). The unique prefix is stored in Redis, and the automatic namespacing is cluster friendly. If the prefix is lost for any reason (expired, or Redis restarts) then a new one will be generated automatically and content will be re-cached by Cerb as it is retrieved again. [Cache/Platform] Cache engines now declare whether they support volatile content with an isVolatile() method. This allows Cerb to adapt to the type of caching being used. For instance, thousands of cache files won't be created in the local filesystem since this can create I/O bottlenecks (particularly when keys are frequently written, since reads are likely efficiently cached in memory by the OS already). However, hundreds of thousands of cache keys could be written to Redis, Memcached, APC, or XCache without an I/O bottleneck. [Performance/Search/Cache] The results of full-text searches are now cached. With a volatile cache engine (Redis, Memcached) they are cached for 5 minutes, and otherwise they expire at the end of the current request. This solves the long-standing issue where full-text searches were being performed twice when subtotals were enabled. In 6.7+, full-text search filter run first, so their results can be cached consistently regardless of other factors. This means that if more than one worker searches for the same thing, the intermediate results are shared, even if the rest of their worklist filters differ. Additionally, when a worker changes pages, and adds or removes filters, the expensive search no longer needs to run again. This is a major performance improvement, since most logged 'slow queries' are the result of full-text searches being repeated (by subtotals, and by workers changing other filters). For the best performance, you should enable a memory-based cache engine from Setup->Configure->Cache. [Cache/Memcached/Platform] The Memcached cache engine now generates a unique key prefix every time the cache is globally cleared. This effectively namespaces keys so that old keys will become inactive and be automatically cleaned up. This is proxy- and multi-tenant friendly since the 'flush' command is never used. Objects are now given a default time-to-live of 24 hours to help facilitate this. This approach enables a lot of dynamic content to be cached without concern over exhausting the available memory (e.g. searches, records, worklists). The unique prefix is stored in Memcached, and the automatic namespacing is cluster friendly. If the prefix is lost for any reason (expired, or Memcached restarts) then a new one will be generated automatically and content will be re-cached by Cerb as it is retrieved again. [Performance/Virtual Attendants] Previously, the target record of a Virtual Attendant event (like "New mail received by group") was always pre-loaded before the behaviors ran. Even though the behaviors efficiently shared the same record while running, this was still inefficient when no behaviors existed on the event point, because the data was still loaded anyway. The target record is now only "lazy loaded" from the database when it is first requested (which means at least one active Virtual Attendant behavior is watching the event). This change sped up the email parser by 200-300% during development profiling, and it should improve the performance of everywhere VAs are used in Cerb. [Performance/Virtual Attendants] Virtual Attendant 'contexts' (the available fields and labels of a related set of records) are now cached in-memory for the duration of a request. This is particularly efficient in the parser, where the same VA behaviors might run hundreds of times in one /cron request. These contexts generally don't take long to build, but there is no reason to repeat that process hundreds of times when the cost of temporarily storing them in memory is negligible. This mainly saves CPU cycles. [CHD-3321] [UI/Usability] All tabs that load dynamic content from the server now show a loading "spinner" image while waiting for a response. This gives workers a visual cue that something is happening in the background. Previously, tabs that were loading were either blank or they showed their outdated content until the new content arrived. This improvement is especially helpful on high latency connections. [CHD-3640] [UI/Search/Usability] When performing a quick search, clicking on one of the examples in the tooltip will now automatically place it into the text field. This time-saving shortcut is particularly useful for fields like 'status' where a small number of discrete options no longer need to be typed. [CHD-2067] [Security/Logins] Added a unique "salt" to worker password hashes to make them much more difficult to crack if they are ever exposed. [Security/Logins] Password hash information is no longer included in the worker table, model objects, or caches. There is no reason to reference a password hash outside of the login process; and that can involve the database directly, where the results can be discarded afterwards. The more places that password hashes are stored, the more vulnerable they are. [Security/Logins] Fixed an issue where password hash comparisons were taking place in MySQL rather than in PHP. This could unnecessarily leave password traces in MySQL log files and the process list. [Performance/Cache] Retrieving a worker record by email address now uses the existing worker cache. Previously this was needlessly querying the database. [Security/Logins] All logins, successful or not, are now slightly delayed on purpose. This adds an artificial cost to brute force attacks. If successful logins weren't delayed, smart attackers could just abandon a login attempt if it was taking noticeably longer than a successful login would. In the near future, this delay should be increased systematically for each consecutive login failure. [Security/Mail/Relay] Improved the way that incoming email relay messages are authorized. Previously, the 'Message-ID:' header included the internal ticket ID, and the security hash used the worker's password to sign it. This meant when a worker changed their password the could no longer reply to any pre-existing relay messages. [Mail/Relay] Incoming email relay replies are now associated with the message they were based on. This allows features like response times to be calculated properly. Previously, a relay reply was always associated with the latest message on a ticket. [CHD-2426] [CHD-1993] [Worklists/Export] The export feature on worklists now provides the ability to include all fields and linked records. Previously, only a small subset of the fields were available. For instance, exports can now include custom fields from the ticket's sender's organization. Additionally, large text fields like 'message content' are now available as well. This uses the same underlying functionality as the Web-API, but with the usability of the API. This also fixes the issue where only IDs were available for some fields, which weren't human-readable. A worker may now choose to export IDs, labels, or both. [CHD-2426] [Worklists/Export] When exporting data from a worklist, human-readable labels are now provided for the fields. In CSV format, these are the headings in the first row. For XML and JSON, the labels are provided as a separate object next to the results. Previously, only arcane property names were returned, like 'cf_123'. [CHD-2426] [Worklists/Export] When exporting data from a worklist, any number of fields can be selected now as a simple checklist. Previously the limit was 15. These fields can also be reordered by dragging them into place. [CHD-3337] [Snippets/Worklist/Export] Snippet records and their content are now exportable from worklists in CSV, XML, or JSON formats. [Worklists/Export] When exporting worklist results in JSON or XML format, field types are included along with the labels. These are useful for interpreting the data; for instance, discerning between a timestamp, checkbox, or number. [Performance/VAs/Contexts] The message '_label' placeholder is now lazy loaded when first used. This requires an extra lookup to use the ticket mask in the output, which is inefficient if the placeholder isn't used. This was particularly inefficient in areas like the mobile plugin, API, and Virtual Attendants where there could be a hundred placeholder dictionaries open at the same time. [Performance/VAs/Snippets] Optimized the way placeholder dictionaries lazy-load content. Previously, cycles were being wasted crawling through linked contexts in the dictionary every time a placeholder was loaded. For instance, if requesting 'ticketlatestmessagesenderfullname' and 'ticketlatestmessagesender_address', data was being recomputed that could have been cached when the dictionary was created. This only cost a few milliseconds, but in some areas that code can run many times (worklist export, API, VA, mobile). [Performance/Platform/Contexts] Optimized the way multiple placeholder dictionaries retrieve their values. The feature was originally designed to be used by Virtual Attendants for a single target, but it now powers the mobile UI, web API, Virtual Attendant list variables, and worklist exports. Previously, each dictionary loaded its values independently, which created a lot of redundant database queries (e.g. multiple tickets loading the same organization or sender information). Now the dictionaries are first analyzed to see what linked records they have in common, and only distinct records are loaded. Duplicates records are served from a high-speed temporary cache. [Worklist/Export/JSON/Performance] Worklist exports in JSON format are now streamed so that peak memory usage remains low. Previously, all the records loaded into memory before they were displayed. [Worklist/Export/XML/Performance] Worklist exports in XML format are now streamed so that peak memory usage remains low. Previously, all the records loaded into memory before they were displayed. [Worklists/Export/Usability] When exporting a worklist to CSV, JSON, or XML, a picker menu is now displayed to quickly search for and select any number of desired fields. Previously, fields had to be selected from a fixed list of 15 dropdowns. The order of the exported fields can be rearranged by dragging them into place. [Worklists/Export/Usability] When exporting a worklist, an option is now provided to automatically convert timestamps into a human readable format (e.g. "Sun, 06 Apr 2014 02:58:03 -0700"). Previously, timestamps were always in Unix epoch format (e.g. 1396778283), which complicated working with the exported data in programs like Excel. [Worklist/Export] Worklist exports now generate output incrementally while displaying the current progress in the browser. This helps to avoid the time and memory constraints inherent in long-running processes. When an export is complete, a download link is shown. Previously, exports attempted to run in a single request, which would become unsuccessful after a list reached a certain size. [Worklists/Export/Messages] Added 'export' functionality to message-based worklists. [Worklists/Export/Notifications] Added 'export' functionality to notification-based worklists. [Security/Cache/Filesystem] The filesystem cache now appends a unique suffix to the generated file names in the storage/tmp/ directory. This helps protect cache file contents in misconfigured environments where the storage/ directory contents are readable by web browsers, since the files no longer have predictable names like 'cache--chworkersall'. This is simply a last line of defense against an attacker scanning for sensitive info in the cache, and it doesn't provide any protection if the web server is very misconfigured to provide a directory index of all the cache files. Make sure you web server is blocking access to all these directories (api/, features/, libs/, storage/). [Less]

Release notes for Cerb 6.6 Cerb (6.6) is a major functionality update released on January 31, 2014. It contains over 70 new features and usability tweaks from community feedback. See: http://wiki.cerbweb.com/6.6 Core [Platform/Custom Fields] ... [More] Implemented configurable parameters for custom fields. The various custom fields types can now provide additional options to modify how they work. Picklist and multi-checkbox fields had a hardcoded parameter for their option list which has been converted to the new format. [CHD-3582] [Custom Fields/Owners] Added a 'Send worker notifications' option to worker-based custom fields. This makes it easy to add an 'owner' field to any kind of record. More advanced workflows are also possible; for example, having two owner fields for a 'salesperson' and 'sales support'. Owners receive watcher notifications about record activity for the duration of their assignment. They don't need to explicitly watch or unwatch records. [Custom Fields/Links] Implemented a new 'Record Link' custom field type. This is similar to how the Links tab works, except the custom field is formalized and the fields of the linked record will be shown in Virtual Attendant behaviors, snippets, etc. For instance, a time tracking entry could have a custom fieldset with fields for an organization and a domain. Time tracking records could then be filtered based on those links, and the related record information could be used in Virtual Attendants, worklists, and the API. [Setup/Storage/Attachments] Fixed an issue where an attachment worklist with multiple links pointing to the same file resulted in missing rows. [Setup/Attachments/Worklists] Attachment worklists now display the storage profile column as a name rather than an ID. [CHD-3377] [Setup/Attachments/Worklists] Attachment worklists once again provide links to the records containing the files (i.e. the 'Context' column). Previously, a link was provided to the ticket but workers had to dig for the message containing the file. Now, when clicking on the permalink for an email message attachment that specific message will be expanded and focused. [CHD-2971] [Mail/Reply/Usability/Keyboard] When using the keyboard to navigate the ticket reply form, using the attachments upload popup previously broke the TAB flow through the form. Now the focus is returned to the attachments (+) button after files have been uploaded. [Storage/Platform/Optimization/Performance] Cerb now automatically detects when a new storage object duplicates an existing one, and an additional link will be created to the same underlying file. A file is only removed when there are zero links pointing at it (e.g. message or comment attachments). Duplicate attachment detection is implemented for incoming mail, worker replies through the UI, the worker proxy, messages imported from an ImpEx export, and files uploaded through the API. This should save a considerable amount of storage space in environments where the same attachments (e.g. PDFs, forms, ebooks) are sent by workers on a regular basis. Similarly, when the same senders include logos and other content in their signature these can be condensed into a single storage object. Currently, existing storage objects are not hashed since would be very intensive on busy systems. New duplicates will be prevented from version 6.6 onward. However, we'll be providing instructions on how to retroactively hash existing storage objects and remove redundancy (technical note: this involves providing SHA1 hashes in the attachment.storage_sha1hash database column, and redirecting attachment_link records based on that). [Mail/Parser/Performance] The original_message.html attachment is now created more efficiently for new incoming messages. Previously its contents were written to a temporary file that was then read into long-term storage. Now it is saved directly from memory into storage. This improvement also made it easier to modify the contents of the HTML message to improve usability (e.g. rewriting links for inline attachments to Cerb URLs). [Mail/Parser/Usability] When viewing an original_message.html attachment in the browser, inline images will now be properly displayed in the content. For instance, this is useful when customers send screenshots by embedding them in the middle of their reply. Previously, such images were included as attachments on the ticket but they weren't displayed in the HTML contents (technical note: The IMG tags still referenced the MIME content-id header). These IMG links are now rewritten to use Cerb file paths. This process works for all new messages that are received in version 6.6 or later, but it will not fix existing original_message.html attachments. [Devblocks/Platform/Markdown] Added the Parsedown library http://parsedown.org/ to Devblocks. This is a faster, leaner, and more modern replacement for the existing PHP Markdown Extra library we've been using up to this point. [Setup/Storage/S3/Usability] When testing a new S3 storage engine profile, a temporary object will be saved, read, and deleted to verify that the credentials work for all actions. Previously the contents of the bucket were listed, but this didn't verify that content could actually be stored and retrieved. [Setup/Storage/S3/Usability] The S3 storage engine now supports a 'Path prefix' configuration option. Previously all content was saved starting at the top-level of a bucket. Now a single bucket can be shared between multiple instances of Cerb. [Setup/Storage/S3/Security] When configuring the S3 storage engine, the secret key parameter is no longer displayed in the profile popup form. If left blank it will remain unchanged. This is far more secure than transmitting it. [Storage/Performance] The storage service now has the ability to queue multiple delete operations to perform them in a single transaction. This is useful for storage engines where incremental deletes would be very expensive. For instance, S3 would generate an HTTP request per file being deleted, which could potentially be hundreds or thousands. Waiting for that many HTTP requests during cron.maint is undesirable. It is now possible for hundreds of objects to be deleted in a single request. [Setup/Storage/S3/Usability] Implemented batch delete functionality in the S3 storage engine. Up to 500 objects can be deleted from S3 in a single HTTP request. [CHD-2711] [Mail/Relay/Comments] When a worker is replying to a relayed message from an external email account, the #comment tag will now allow comments to span multiple lines. The comment will be terminated when it encounters a blank line or a line starting with another #tag. This was necessary because some email applications, like Gmail, force text/plain content to hard wrap on spaces before 76 characters, and quoted-printable encoding isn't used to preserve the original wrapping. The comment will still appear as a single paragraph in Cerb. [CHD-2711] [Mail/Relay/Comments] A new #start comment tag is available to workers that respond to relayed messages through an external email account. This allows multiple line comments where linefeeds will be preserved. This is especially useful for chunks of formatted text like code fragments and patches. A multiple line comment is terminated with an #end tag on its own line. If a worker's mail application adds hard linefeeds to their comment content, those will be preserved in the comment created by Cerb. [Mail/Parser/Localization] Cerb will now use the iconv PHP extension if it is loaded. This is capable of converting a wider range of encodings than mbstring. For instance, TIS-620 (Thai) and WIN-1258 (Vietnamese) were not properly converted to Unicode by Cerb when only using mbstring. [Mail/Relay] Workers can now manually relay specific messages to external email accounts without having to set up a Virtual Attendant behavior first. The reply menu on each message contains a new 'Relay to worker email' option. Selecting the relay option displays a popup where worker email accounts can be filtered and selected. This feature is particularly useful for situations where workers may need to reply while offline, such as during transoceanic flights or cruises. A worker can relay messages from Cerb to their mobile email application, reply at their leisure, and their messages will be delivered when network access is restored. The feature is also useful when certain workers handle requests through email without logging into the Cerb interface (e.g. temps, interns, consultants). A human dispatcher can route messages to these workers on an as-needed basis. [Mail/Relay/Activity Log] An Activity Log entry is now created when messages are relayed to an email account by a worker or Virtual Attendant. This notifies other workers that work may be taking place on the ticket from outside of Cerb, and it also serves to keep a better history of the actions of Virtual Attendants. Previously, there was no record on a ticket that it had been relayed unless a comment was explicitly created to say so. [Mail/Activity Log/Preferences] Workers can now enable the display of Activity Log entries on the ticket timeline. This displays a mix of messages, comments, and activity log data in a single place without having to click into the 'Activity Log' tab. The option is located on the Settings page that is accessed from the menu displayed when a worker clicks on their name in the top right of the UI. Consecutive log entries will be combined in a single block, and those that take place with a similar timestamp are grouped together for readability. This option is also compatible with the 'Read all' option where the timeline is displayed expanded in 'oldest first' order rather than 'newest first'. [Devblocks/Platform/jQuery] Updated jQuery dependency from 1.7.2 to 1.10.2 [Devblocks/Platform/jQuery] Updated jQuery UI dependency from 1.9.1 to 1.10.3 [Devblocks/Platform/Dependencies] Added the jQuery Caret plugin for getting and setting the cursor position in textareas. [Devblocks/Platform/jqPlot] Updated jqplot dependency from 1.0.0b2 to 1.0.8 [Devblocks/Platform/qTip] Updated qTip dependency from 1.0.0rc3 to 2.2.0 [Tour/Usability] Updated and expanded the tour so it provides tips and callouts about the main functionality within Cerb. This should help onboard new workers faster. [Tour/Usability] When clicking on a new 'Point of Interest' callout in tour mode, the previously viewed callout will now be closed automatically. [CHD-786] [Mail/HTML] Workers can now send HTML formatted email replies from Cerb. This feature was a long time coming, as we wanted to implement it in a way that was consistent with Cerb's spirit of productivity and automation, which are heavily based on the efficient processing of large amounts of text. To facilitate this, the feature is divided into two parts: HTML templates and Markdown syntax. Multiple HTML templates (e.g. letterhead and stylesheets) can be created from the Setup->Mail->HTML page. Each group and bucket can be configured to use a specific template by default, which determines the style of their HTML replies. Templates can contain any valid HTML, although most mail apps will sanitize the messages (e.g. removing Javascript). Rather than introducing a visual HTML editor which obscures the underlying text, workers can reply with Markdown syntax to bold, italicize, link, embed images, quote, and to add headings, code blocks, and lists. Markdown is a markup language that was designed for use in email to preserve human readability. This "progressive enhancement" approach provides workers with greater flexibility in their replies without compromising efficiency. Outgoing messages in the Cerb history are still displayed as plaintext. Additionally, Virtual Attendant behaviors are capable of working with Markdown syntax much easier than with HTML directly. [CHD-786] [Mail/Reply/HTML] When workers are replying to mail, an editor is now provided to assist with using Markdown formatting. By default the editor starts in plaintext mode, which behaves the same as Cerb always has (i.e. only sending text responses). In this case the toolbar only displays a single button that toggles rich text functionality. Once that button is clicked, the toolbar expands to provide: bold, italics, unordered and ordered lists, embedded images, external images, links, quotes, code blocks, and a preview option. The preview option displays the HTML version of the current reply in a popup window. It will also use the HTML template of the current group and bucket, if available. The preview window can be left open while responding, and it will update its contents every time the RETURN key is pressed. [Storage/Files] Attachments can now be requested by their SHA-1 hash in a URL, in addition to the globally unique ID of any of their links. This makes it possible to refer to attachments that have been uploaded but not linked to anything yet. For instance, this is useful for implementing embedded images in mail and knowledgebase articles; those images would need to be previewed before the record (and link) is created. [Platform/Usability] Popups will now open in the middle of the browser window by default. They previously opened in the top center, which made smaller popups less noticeable. [Plugins/Platform] When a file is uploaded using the file chooser popup, it now provides the caller with a SHA-1 hash and internal URL, as well as the entire response object (name, size, mime type, etc). This should make the file chooser reusable in more situations; for instance, in providing embedded image functionality. [Mail/Reply/HTML] Workers can now send inline images in their replies. A new 'Upload an Image' button is available in the toolbar when replying to mail with HTML formatting. This button displays the file chooser popup, which automatically generates the syntax for an embedded image once saved. The image is visible in the preview popup, and once the message is sent Cerb will automatically convert it to an embedded image for the recipient (i.e. replacing the link to Cerb with a self-contained MIME part). [Mail/Search/Usability] When using Search->Ticket, the results will no longer be restricted to only a worker's own groups. The group filter is now added by default but it can be removed from the editable filters. Previously it was a required filter that couldn't be removed. This has been requested several times by organizations that want workers to see the "Access denied" listing for a ticket when searching by mask or participant, instead of having zero results shown (which leads people to believe they're doing the search wrong). [Platform/Popups/Usability] Popups will now float over the same position in the browser even when the rest of the page scrolls. Previously, popups could be scrolled off the page and become lost. [Mail/Peek/Usability] When viewing the peek popup for a ticket, its group and bucket are now shown as two linked dropdowns. Changing the group in the first dropdown will change the list of available buckets in the second one. Previously, the group and bucket were only visible by clicking the dropdown and looking for the (*) marker, which was cumbersome. Additionally, the previous dropdown also combined groups and buckets into a single long list which was difficult to use. [Mail/Compose/Usability] When composing mail, the 'From:' option now displays the group and bucket in two linked dropdowns instead of one big list. [Mail/Compose/HTML] Added HTML message support to the compose popup. The preview feature will display the template for the currently selected group and bucket. [Storage/Files] The '/files' page now provides cache control headers for the content it serves. This allows a worker's browser to cache certain resources (like images) after their first retrieval so they aren't requested from the server again. This is useful now that content like embedded images in HTML messages are displayed through this page. [Platform/Dependencies] Updated HTMLPurifier library from 4.5 to 4.6. [Platform/Devblocks/Dependencies] Added the 'CSS to Inline Styles' library by Tijs Verkoyen. This provides an easy to way to convert CSS STYLE blocks to inline styles when sending HTML email, as many email readers filter out HEAD, BODY, and STYLE. [Mail/Reply/HTML/Preferences] Added a worker preference for "Always switch to HTML mode" while replying to mail. [Platform/Plugins/Popups] Plugins that use genericAjaxPopup() can now specify the 'target' option using a jQuery Position object; e.g. { my: 'top left', at: 'bottom right', of: 'selector' }. [CHD-3622] [Explore/Usability] Improved the way the last accessed time is recorded for explorer sets. Previously the access timestamp only updated 20% of the time in order to reduce extraneous UPDATE database queries when someone quickly paged through the list. Now, instead, the access time is updated any time at least 30 seconds has elapsed since the last recorded access time. Additionally, the access time is now always recorded when the set is created. The prior inefficiency rarely caused problems, but it would manifest if an explorer set was created shortly before the maintenance scheduled job ran, or if the maintenance job was set to run very often instead of nightly. This could result in explorer sets being purged while they were still in use. [Mail/HTML/Broadcast] Implemented HTML message support in ticket worklist broadcasts from bulk update. [Mail/HTML/Broadcast] Implemented HTML message support in address worklist broadcasts from bulk update. [Mail/HTML/Broadcast] Implemented HTML message support in opportunity worklist broadcasts from bulk update. [Mail/Usability/HTML] When generating a plaintext part from an HTML-only message, Cerb now attempts to preserve whitespace in pre-formatted (PRE) and CODE blocks. [Mail/Usability/HTML] When generating a plaintext part from an HTML-only message, Cerb now attempts to prefix nested BLOCKQUOTE content with '>'. [Mail/HTML/Usability] When sending HTML mail, Cerb will generate a new plaintext part that cleans up Markdown formatting. This plaintext part is sent along with the HTML message (e.g. for mobile and automated mail readers), and it's also what's stored in the conversation history. [REVERT] [Platform/Popups/Usability] Popups no longer open up in the center of the browser window by default. This was causing problems on popups that grow, since they could expand beyond the bottom of the window and constantly require being dragged into place. Popups now display at the top so they have the full browser height to grow before they show a scrollbar. When a popup is opened a position can be defined now, so popups that don't grow (like the file chooser) may still elect to show up in the center of the page. [Setup/Mail/Reply-To/HTML] Reply-to addresses can now specify a default HTML template for outgoing mail. The template set on the default reply-to address is the global default. When a group doesn't have its own HTML template it will default to the one on its reply-to address. [Mail/HTML/Usability] The upgrade script for 6.6 now automatically creates a default HTML template and associates it with the default reply-to address. This allows HTML replies to "just work" without any manual configuration. [Virtual Attendants/Mail/HTML] The 'Send mail' action in Virtual Attendant behaviors now supports sending messages using Markdown and HTML templates. [Virtual Attendants/Mail/HTML] The 'Send mail to recipients' action in Virtual Attendant behaviors now supports sending messages using Markdown and HTML templates. [Mail/HTML/Reply] Workers can now optionally choose a specific HTML template when replying. If nothing is selected, the current group/bucket's default is used. [Mail/HTML/Compose] Workers can now optionally choose a specific HTML template when composing a new message. If nothing is selected, the given group/bucket's default is used. [Mail/HTML/Broadcast/Tickets] Workers can now optionally choose a specific HTML template when broadcasting from ticket worklists. If nothing is selected, the default for the group/bucket is used instead. [Mail/HTML/Broadcast/Addresses] Workers can now optionally choose a specific HTML template when broadcasting from email address worklists. If nothing is selected, the default for the given group/bucket is used instead. [Mail/HTML/Broadcast/CRM/Opps] Workers can now optionally choose a specific HTML template when broadcasting from opportunity worklists. If nothing is selected, the default for the group/bucket is used instead. [Mail/Reply/Usability] When replying to a message, the "Would you like to move this conversation?" option now shows the groups and buckets in two linked dropdowns. Changing the group in the first list automatically displays the available buckets in the second list. Previously, the current group and bucket was displayed as a default "No, leave it in the [name] bucket of [group]" option, and moving the conversation required using a long combined list of all the buckets which was difficult to navigate. The new way is also more efficient when using keyboard shortcuts. [Performance/Worklists/Tickets] When using the "Is in groups of [worker]" filter on ticket worklists, the database query is now optimized if the worker is a member of every group. Previously, this inefficiently checked the group_id on every result anyway. [Support Center/Contact/Usability] The ability to upload multiple files in the Support Center now uses a browser's HTML5 support. [Support Center/Contact/Usability] The file type is now set properly on attachments uploaded through the Support Center. Previously, all uploaded files were being saved as 'application/octet-stream', which forced attachments to download even if they could be displayed in the browser (e.g. images, text files). [Web-API/Contexts] Implemented 'GET /rest/contexts/list.json' in the Web-API for retrieving a list of context IDs and names. The context IDs are used for making various kinds of requests (e.g. attachment links, comments, record links) and there wasn't a comprehensive list available anywhere. This API request will also include any contexts that were added through plugins, which the standard documentation wouldn't include. [CHD-3562] [Web-API/Contexts/Links] Implemented 'POST /rest/contexts/link.json' in the Web-API for adding any number of links to a record. See: http://cerbweb.com/book/latest/developer_guide/rest_api/contexts.html [CHD-3562] [Web-API/Contexts/Links] Implemented 'POST /rest/contexts/unlink.json' in the Web-API for removing any number of links from a record. See: http://cerbweb.com/book/latest/developer_guide/rest_api/contexts.html [Setup/Groups] Fixed an issue when deleting a group where group-owned Virtual Attendants and custom fieldsets may not be cleaned up properly. [Less]

Release notes for Cerb 6.6 Cerb (6.6) is a major functionality update released on January 31, 2014. It contains over 70 new features and usability tweaks from community feedback. See: http://wiki.cerbweb.com/6.6 Core [Platform/Custom Fields] ... [More] Implemented configurable parameters for custom fields. The various custom fields types can now provide additional options to modify how they work. Picklist and multi-checkbox fields had a hardcoded parameter for their option list which has been converted to the new format. [CHD-3582] [Custom Fields/Owners] Added a 'Send worker notifications' option to worker-based custom fields. This makes it easy to add an 'owner' field to any kind of record. More advanced workflows are also possible; for example, having two owner fields for a 'salesperson' and 'sales support'. Owners receive watcher notifications about record activity for the duration of their assignment. They don't need to explicitly watch or unwatch records. [Custom Fields/Links] Implemented a new 'Record Link' custom field type. This is similar to how the Links tab works, except the custom field is formalized and the fields of the linked record will be shown in Virtual Attendant behaviors, snippets, etc. For instance, a time tracking entry could have a custom fieldset with fields for an organization and a domain. Time tracking records could then be filtered based on those links, and the related record information could be used in Virtual Attendants, worklists, and the API. [Setup/Storage/Attachments] Fixed an issue where an attachment worklist with multiple links pointing to the same file resulted in missing rows. [Setup/Attachments/Worklists] Attachment worklists now display the storage profile column as a name rather than an ID. [CHD-3377] [Setup/Attachments/Worklists] Attachment worklists once again provide links to the records containing the files (i.e. the 'Context' column). Previously, a link was provided to the ticket but workers had to dig for the message containing the file. Now, when clicking on the permalink for an email message attachment that specific message will be expanded and focused. [CHD-2971] [Mail/Reply/Usability/Keyboard] When using the keyboard to navigate the ticket reply form, using the attachments upload popup previously broke the TAB flow through the form. Now the focus is returned to the attachments (+) button after files have been uploaded. [Storage/Platform/Optimization/Performance] Cerb now automatically detects when a new storage object duplicates an existing one, and an additional link will be created to the same underlying file. A file is only removed when there are zero links pointing at it (e.g. message or comment attachments). Duplicate attachment detection is implemented for incoming mail, worker replies through the UI, the worker proxy, messages imported from an ImpEx export, and files uploaded through the API. This should save a considerable amount of storage space in environments where the same attachments (e.g. PDFs, forms, ebooks) are sent by workers on a regular basis. Similarly, when the same senders include logos and other content in their signature these can be condensed into a single storage object. Currently, existing storage objects are not hashed since would be very intensive on busy systems. New duplicates will be prevented from version 6.6 onward. However, we'll be providing instructions on how to retroactively hash existing storage objects and remove redundancy (technical note: this involves providing SHA1 hashes in the attachment.storage_sha1hash database column, and redirecting attachment_link records based on that). [Mail/Parser/Performance] The original_message.html attachment is now created more efficiently for new incoming messages. Previously its contents were written to a temporary file that was then read into long-term storage. Now it is saved directly from memory into storage. This improvement also made it easier to modify the contents of the HTML message to improve usability (e.g. rewriting links for inline attachments to Cerb URLs). [Mail/Parser/Usability] When viewing an original_message.html attachment in the browser, inline images will now be properly displayed in the content. For instance, this is useful when customers send screenshots by embedding them in the middle of their reply. Previously, such images were included as attachments on the ticket but they weren't displayed in the HTML contents (technical note: The IMG tags still referenced the MIME content-id header). These IMG links are now rewritten to use Cerb file paths. This process works for all new messages that are received in version 6.6 or later, but it will not fix existing original_message.html attachments. [Devblocks/Platform/Markdown] Added the Parsedown library http://parsedown.org/ to Devblocks. This is a faster, leaner, and more modern replacement for the existing PHP Markdown Extra library we've been using up to this point. [Setup/Storage/S3/Usability] When testing a new S3 storage engine profile, a temporary object will be saved, read, and deleted to verify that the credentials work for all actions. Previously the contents of the bucket were listed, but this didn't verify that content could actually be stored and retrieved. [Setup/Storage/S3/Usability] The S3 storage engine now supports a 'Path prefix' configuration option. Previously all content was saved starting at the top-level of a bucket. Now a single bucket can be shared between multiple instances of Cerb. [Setup/Storage/S3/Security] When configuring the S3 storage engine, the secret key parameter is no longer displayed in the profile popup form. If left blank it will remain unchanged. This is far more secure than transmitting it. [Storage/Performance] The storage service now has the ability to queue multiple delete operations to perform them in a single transaction. This is useful for storage engines where incremental deletes would be very expensive. For instance, S3 would generate an HTTP request per file being deleted, which could potentially be hundreds or thousands. Waiting for that many HTTP requests during cron.maint is undesirable. It is now possible for hundreds of objects to be deleted in a single request. [Setup/Storage/S3/Usability] Implemented batch delete functionality in the S3 storage engine. Up to 500 objects can be deleted from S3 in a single HTTP request. [CHD-2711] [Mail/Relay/Comments] When a worker is replying to a relayed message from an external email account, the #comment tag will now allow comments to span multiple lines. The comment will be terminated when it encounters a blank line or a line starting with another #tag. This was necessary because some email applications, like Gmail, force text/plain content to hard wrap on spaces before 76 characters, and quoted-printable encoding isn't used to preserve the original wrapping. The comment will still appear as a single paragraph in Cerb. [CHD-2711] [Mail/Relay/Comments] A new #start comment tag is available to workers that respond to relayed messages through an external email account. This allows multiple line comments where linefeeds will be preserved. This is especially useful for chunks of formatted text like code fragments and patches. A multiple line comment is terminated with an #end tag on its own line. If a worker's mail application adds hard linefeeds to their comment content, those will be preserved in the comment created by Cerb. [Mail/Parser/Localization] Cerb will now use the iconv PHP extension if it is loaded. This is capable of converting a wider range of encodings than mbstring. For instance, TIS-620 (Thai) and WIN-1258 (Vietnamese) were not properly converted to Unicode by Cerb when only using mbstring. [Mail/Relay] Workers can now manually relay specific messages to external email accounts without having to set up a Virtual Attendant behavior first. The reply menu on each message contains a new 'Relay to worker email' option. Selecting the relay option displays a popup where worker email accounts can be filtered and selected. This feature is particularly useful for situations where workers may need to reply while offline, such as during transoceanic flights or cruises. A worker can relay messages from Cerb to their mobile email application, reply at their leisure, and their messages will be delivered when network access is restored. The feature is also useful when certain workers handle requests through email without logging into the Cerb interface (e.g. temps, interns, consultants). A human dispatcher can route messages to these workers on an as-needed basis. [Mail/Relay/Activity Log] An Activity Log entry is now created when messages are relayed to an email account by a worker or Virtual Attendant. This notifies other workers that work may be taking place on the ticket from outside of Cerb, and it also serves to keep a better history of the actions of Virtual Attendants. Previously, there was no record on a ticket that it had been relayed unless a comment was explicitly created to say so. [Mail/Activity Log/Preferences] Workers can now enable the display of Activity Log entries on the ticket timeline. This displays a mix of messages, comments, and activity log data in a single place without having to click into the 'Activity Log' tab. The option is located on the Settings page that is accessed from the menu displayed when a worker clicks on their name in the top right of the UI. Consecutive log entries will be combined in a single block, and those that take place with a similar timestamp are grouped together for readability. This option is also compatible with the 'Read all' option where the timeline is displayed expanded in 'oldest first' order rather than 'newest first'. [Devblocks/Platform/jQuery] Updated jQuery dependency from 1.7.2 to 1.10.2 [Devblocks/Platform/jQuery] Updated jQuery UI dependency from 1.9.1 to 1.10.3 [Devblocks/Platform/Dependencies] Added the jQuery Caret plugin for getting and setting the cursor position in textareas. [Devblocks/Platform/jqPlot] Updated jqplot dependency from 1.0.0b2 to 1.0.8 [Devblocks/Platform/qTip] Updated qTip dependency from 1.0.0rc3 to 2.2.0 [Tour/Usability] Updated and expanded the tour so it provides tips and callouts about the main functionality within Cerb. This should help onboard new workers faster. [Tour/Usability] When clicking on a new 'Point of Interest' callout in tour mode, the previously viewed callout will now be closed automatically. [CHD-786] [Mail/HTML] Workers can now send HTML formatted email replies from Cerb. This feature was a long time coming, as we wanted to implement it in a way that was consistent with Cerb's spirit of productivity and automation, which are heavily based on the efficient processing of large amounts of text. To facilitate this, the feature is divided into two parts: HTML templates and Markdown syntax. Multiple HTML templates (e.g. letterhead and stylesheets) can be created from the Setup->Mail->HTML page. Each group and bucket can be configured to use a specific template by default, which determines the style of their HTML replies. Templates can contain any valid HTML, although most mail apps will sanitize the messages (e.g. removing Javascript). Rather than introducing a visual HTML editor which obscures the underlying text, workers can reply with Markdown syntax to bold, italicize, link, embed images, quote, and to add headings, code blocks, and lists. Markdown is a markup language that was designed for use in email to preserve human readability. This "progressive enhancement" approach provides workers with greater flexibility in their replies without compromising efficiency. Outgoing messages in the Cerb history are still displayed as plaintext. Additionally, Virtual Attendant behaviors are capable of working with Markdown syntax much easier than with HTML directly. [CHD-786] [Mail/Reply/HTML] When workers are replying to mail, an editor is now provided to assist with using Markdown formatting. By default the editor starts in plaintext mode, which behaves the same as Cerb always has (i.e. only sending text responses). In this case the toolbar only displays a single button that toggles rich text functionality. Once that button is clicked, the toolbar expands to provide: bold, italics, unordered and ordered lists, embedded images, external images, links, quotes, code blocks, and a preview option. The preview option displays the HTML version of the current reply in a popup window. It will also use the HTML template of the current group and bucket, if available. The preview window can be left open while responding, and it will update its contents every time the RETURN key is pressed. [Storage/Files] Attachments can now be requested by their SHA-1 hash in a URL, in addition to the globally unique ID of any of their links. This makes it possible to refer to attachments that have been uploaded but not linked to anything yet. For instance, this is useful for implementing embedded images in mail and knowledgebase articles; those images would need to be previewed before the record (and link) is created. [Platform/Usability] Popups will now open in the middle of the browser window by default. They previously opened in the top center, which made smaller popups less noticeable. [Plugins/Platform] When a file is uploaded using the file chooser popup, it now provides the caller with a SHA-1 hash and internal URL, as well as the entire response object (name, size, mime type, etc). This should make the file chooser reusable in more situations; for instance, in providing embedded image functionality. [Mail/Reply/HTML] Workers can now send inline images in their replies. A new 'Upload an Image' button is available in the toolbar when replying to mail with HTML formatting. This button displays the file chooser popup, which automatically generates the syntax for an embedded image once saved. The image is visible in the preview popup, and once the message is sent Cerb will automatically convert it to an embedded image for the recipient (i.e. replacing the link to Cerb with a self-contained MIME part). [Mail/Search/Usability] When using Search->Ticket, the results will no longer be restricted to only a worker's own groups. The group filter is now added by default but it can be removed from the editable filters. Previously it was a required filter that couldn't be removed. This has been requested several times by organizations that want workers to see the "Access denied" listing for a ticket when searching by mask or participant, instead of having zero results shown (which leads people to believe they're doing the search wrong). [Platform/Popups/Usability] Popups will now float over the same position in the browser even when the rest of the page scrolls. Previously, popups could be scrolled off the page and become lost. [Mail/Peek/Usability] When viewing the peek popup for a ticket, its group and bucket are now shown as two linked dropdowns. Changing the group in the first dropdown will change the list of available buckets in the second one. Previously, the group and bucket were only visible by clicking the dropdown and looking for the (*) marker, which was cumbersome. Additionally, the previous dropdown also combined groups and buckets into a single long list which was difficult to use. [Mail/Compose/Usability] When composing mail, the 'From:' option now displays the group and bucket in two linked dropdowns instead of one big list. [Mail/Compose/HTML] Added HTML message support to the compose popup. The preview feature will display the template for the currently selected group and bucket. [Storage/Files] The '/files' page now provides cache control headers for the content it serves. This allows a worker's browser to cache certain resources (like images) after their first retrieval so they aren't requested from the server again. This is useful now that content like embedded images in HTML messages are displayed through this page. [Platform/Dependencies] Updated HTMLPurifier library from 4.5 to 4.6. [Platform/Devblocks/Dependencies] Added the 'CSS to Inline Styles' library by Tijs Verkoyen. This provides an easy to way to convert CSS STYLE blocks to inline styles when sending HTML email, as many email readers filter out HEAD, BODY, and STYLE. [Mail/Reply/HTML/Preferences] Added a worker preference for "Always switch to HTML mode" while replying to mail. [Platform/Plugins/Popups] Plugins that use genericAjaxPopup() can now specify the 'target' option using a jQuery Position object; e.g. { my: 'top left', at: 'bottom right', of: 'selector' }. [CHD-3622] [Explore/Usability] Improved the way the last accessed time is recorded for explorer sets. Previously the access timestamp only updated 20% of the time in order to reduce extraneous UPDATE database queries when someone quickly paged through the list. Now, instead, the access time is updated any time at least 30 seconds has elapsed since the last recorded access time. Additionally, the access time is now always recorded when the set is created. The prior inefficiency rarely caused problems, but it would manifest if an explorer set was created shortly before the maintenance scheduled job ran, or if the maintenance job was set to run very often instead of nightly. This could result in explorer sets being purged while they were still in use. [Mail/HTML/Broadcast] Implemented HTML message support in ticket worklist broadcasts from bulk update. [Mail/HTML/Broadcast] Implemented HTML message support in address worklist broadcasts from bulk update. [Mail/HTML/Broadcast] Implemented HTML message support in opportunity worklist broadcasts from bulk update. [Mail/Usability/HTML] When generating a plaintext part from an HTML-only message, Cerb now attempts to preserve whitespace in pre-formatted (PRE) and CODE blocks. [Mail/Usability/HTML] When generating a plaintext part from an HTML-only message, Cerb now attempts to prefix nested BLOCKQUOTE content with '>'. [Mail/HTML/Usability] When sending HTML mail, Cerb will generate a new plaintext part that cleans up Markdown formatting. This plaintext part is sent along with the HTML message (e.g. for mobile and automated mail readers), and it's also what's stored in the conversation history. [REVERT] [Platform/Popups/Usability] Popups no longer open up in the center of the browser window by default. This was causing problems on popups that grow, since they could expand beyond the bottom of the window and constantly require being dragged into place. Popups now display at the top so they have the full browser height to grow before they show a scrollbar. When a popup is opened a position can be defined now, so popups that don't grow (like the file chooser) may still elect to show up in the center of the page. [Setup/Mail/Reply-To/HTML] Reply-to addresses can now specify a default HTML template for outgoing mail. The template set on the default reply-to address is the global default. When a group doesn't have its own HTML template it will default to the one on its reply-to address. [Mail/HTML/Usability] The upgrade script for 6.6 now automatically creates a default HTML template and associates it with the default reply-to address. This allows HTML replies to "just work" without any manual configuration. [Virtual Attendants/Mail/HTML] The 'Send mail' action in Virtual Attendant behaviors now supports sending messages using Markdown and HTML templates. [Virtual Attendants/Mail/HTML] The 'Send mail to recipients' action in Virtual Attendant behaviors now supports sending messages using Markdown and HTML templates. [Mail/HTML/Reply] Workers can now optionally choose a specific HTML template when replying. If nothing is selected, the current group/bucket's default is used. [Mail/HTML/Compose] Workers can now optionally choose a specific HTML template when composing a new message. If nothing is selected, the given group/bucket's default is used. [Mail/HTML/Broadcast/Tickets] Workers can now optionally choose a specific HTML template when broadcasting from ticket worklists. If nothing is selected, the default for the group/bucket is used instead. [Mail/HTML/Broadcast/Addresses] Workers can now optionally choose a specific HTML template when broadcasting from email address worklists. If nothing is selected, the default for the given group/bucket is used instead. [Mail/HTML/Broadcast/CRM/Opps] Workers can now optionally choose a specific HTML template when broadcasting from opportunity worklists. If nothing is selected, the default for the group/bucket is used instead. [Mail/Reply/Usability] When replying to a message, the "Would you like to move this conversation?" option now shows the groups and buckets in two linked dropdowns. Changing the group in the first list automatically displays the available buckets in the second list. Previously, the current group and bucket was displayed as a default "No, leave it in the [name] bucket of [group]" option, and moving the conversation required using a long combined list of all the buckets which was difficult to navigate. The new way is also more efficient when using keyboard shortcuts. [Performance/Worklists/Tickets] When using the "Is in groups of [worker]" filter on ticket worklists, the database query is now optimized if the worker is a member of every group. Previously, this inefficiently checked the group_id on every result anyway. [Support Center/Contact/Usability] The ability to upload multiple files in the Support Center now uses a browser's HTML5 support. [Support Center/Contact/Usability] The file type is now set properly on attachments uploaded through the Support Center. Previously, all uploaded files were being saved as 'application/octet-stream', which forced attachments to download even if they could be displayed in the browser (e.g. images, text files). [Web-API/Contexts] Implemented 'GET /rest/contexts/list.json' in the Web-API for retrieving a list of context IDs and names. The context IDs are used for making various kinds of requests (e.g. attachment links, comments, record links) and there wasn't a comprehensive list available anywhere. This API request will also include any contexts that were added through plugins, which the standard documentation wouldn't include. [CHD-3562] [Web-API/Contexts/Links] Implemented 'POST /rest/contexts/link.json' in the Web-API for adding any number of links to a record. See: http://cerbweb.com/book/latest/developerguide/restapi/contexts.html [CHD-3562] [Web-API/Contexts/Links] Implemented 'POST /rest/contexts/unlink.json' in the Web-API for removing any number of links from a record. See: http://cerbweb.com/book/latest/developerguide/restapi/contexts.html [Setup/Groups] Fixed an issue when deleting a group where group-owned Virtual Attendants and custom fieldsets may not be cleaned up properly. [Less]

Release notes for Cerb 6.6 Cerb (6.6) is a major functionality update released on January 31, 2014. It contains over 70 new features and usability tweaks from community feedback. See: http://wiki.cerbweb.com/6.6 Core [Platform/Custom Fields] ... [More] Implemented configurable parameters for custom fields. The various custom fields types can now provide additional options to modify how they work. Picklist and multi-checkbox fields had a hardcoded parameter for their option list which has been converted to the new format. [CHD-3582] [Custom Fields/Owners] Added a 'Send worker notifications' option to worker-based custom fields. This makes it easy to add an 'owner' field to any kind of record. More advanced workflows are also possible; for example, having two owner fields for a 'salesperson' and 'sales support'. Owners receive watcher notifications about record activity for the duration of their assignment. They don't need to explicitly watch or unwatch records. [Custom Fields/Links] Implemented a new 'Record Link' custom field type. This is similar to how the Links tab works, except the custom field is formalized and the fields of the linked record will be shown in Virtual Attendant behaviors, snippets, etc. For instance, a time tracking entry could have a custom fieldset with fields for an organization and a domain. Time tracking records could then be filtered based on those links, and the related record information could be used in Virtual Attendants, worklists, and the API. [Setup/Storage/Attachments] Fixed an issue where an attachment worklist with multiple links pointing to the same file resulted in missing rows. [Setup/Attachments/Worklists] Attachment worklists now display the storage profile column as a name rather than an ID. [CHD-3377] [Setup/Attachments/Worklists] Attachment worklists once again provide links to the records containing the files (i.e. the 'Context' column). Previously, a link was provided to the ticket but workers had to dig for the message containing the file. Now, when clicking on the permalink for an email message attachment that specific message will be expanded and focused. [CHD-2971] [Mail/Reply/Usability/Keyboard] When using the keyboard to navigate the ticket reply form, using the attachments upload popup previously broke the TAB flow through the form. Now the focus is returned to the attachments (+) button after files have been uploaded. [Storage/Platform/Optimization/Performance] Cerb now automatically detects when a new storage object duplicates an existing one, and an additional link will be created to the same underlying file. A file is only removed when there are zero links pointing at it (e.g. message or comment attachments). Duplicate attachment detection is implemented for incoming mail, worker replies through the UI, the worker proxy, messages imported from an ImpEx export, and files uploaded through the API. This should save a considerable amount of storage space in environments where the same attachments (e.g. PDFs, forms, ebooks) are sent by workers on a regular basis. Similarly, when the same senders include logos and other content in their signature these can be condensed into a single storage object. Currently, existing storage objects are not hashed since would be very intensive on busy systems. New duplicates will be prevented from version 6.6 onward. However, we'll be providing instructions on how to retroactively hash existing storage objects and remove redundancy (technical note: this involves providing SHA1 hashes in the attachment.storage_sha1hash database column, and redirecting attachment_link records based on that). [Mail/Parser/Performance] The original_message.html attachment is now created more efficiently for new incoming messages. Previously its contents were written to a temporary file that was then read into long-term storage. Now it is saved directly from memory into storage. This improvement also made it easier to modify the contents of the HTML message to improve usability (e.g. rewriting links for inline attachments to Cerb URLs). [Mail/Parser/Usability] When viewing an original_message.html attachment in the browser, inline images will now be properly displayed in the content. For instance, this is useful when customers send screenshots by embedding them in the middle of their reply. Previously, such images were included as attachments on the ticket but they weren't displayed in the HTML contents (technical note: The IMG tags still referenced the MIME content-id header). These IMG links are now rewritten to use Cerb file paths. This process works for all new messages that are received in version 6.6 or later, but it will not fix existing original_message.html attachments. [Devblocks/Platform/Markdown] Added the Parsedown library http://parsedown.org/ to Devblocks. This is a faster, leaner, and more modern replacement for the existing PHP Markdown Extra library we've been using up to this point. [Setup/Storage/S3/Usability] When testing a new S3 storage engine profile, a temporary object will be saved, read, and deleted to verify that the credentials work for all actions. Previously the contents of the bucket were listed, but this didn't verify that content could actually be stored and retrieved. [Setup/Storage/S3/Usability] The S3 storage engine now supports a 'Path prefix' configuration option. Previously all content was saved starting at the top-level of a bucket. Now a single bucket can be shared between multiple instances of Cerb. [Setup/Storage/S3/Security] When configuring the S3 storage engine, the secret key parameter is no longer displayed in the profile popup form. If left blank it will remain unchanged. This is far more secure than transmitting it. [Storage/Performance] The storage service now has the ability to queue multiple delete operations to perform them in a single transaction. This is useful for storage engines where incremental deletes would be very expensive. For instance, S3 would generate an HTTP request per file being deleted, which could potentially be hundreds or thousands. Waiting for that many HTTP requests during cron.maint is undesirable. It is now possible for hundreds of objects to be deleted in a single request. [Setup/Storage/S3/Usability] Implemented batch delete functionality in the S3 storage engine. Up to 500 objects can be deleted from S3 in a single HTTP request. [CHD-2711] [Mail/Relay/Comments] When a worker is replying to a relayed message from an external email account, the #comment tag will now allow comments to span multiple lines. The comment will be terminated when it encounters a blank line or a line starting with another #tag. This was necessary because some email applications, like Gmail, force text/plain content to hard wrap on spaces before 76 characters, and quoted-printable encoding isn't used to preserve the original wrapping. The comment will still appear as a single paragraph in Cerb. [CHD-2711] [Mail/Relay/Comments] A new #start comment tag is available to workers that respond to relayed messages through an external email account. This allows multiple line comments where linefeeds will be preserved. This is especially useful for chunks of formatted text like code fragments and patches. A multiple line comment is terminated with an #end tag on its own line. If a worker's mail application adds hard linefeeds to their comment content, those will be preserved in the comment created by Cerb. [Mail/Parser/Localization] Cerb will now use the iconv PHP extension if it is loaded. This is capable of converting a wider range of encodings than mbstring. For instance, TIS-620 (Thai) and WIN-1258 (Vietnamese) were not properly converted to Unicode by Cerb when only using mbstring. [Mail/Relay] Workers can now manually relay specific messages to external email accounts without having to set up a Virtual Attendant behavior first. The reply menu on each message contains a new 'Relay to worker email' option. Selecting the relay option displays a popup where worker email accounts can be filtered and selected. This feature is particularly useful for situations where workers may need to reply while offline, such as during transoceanic flights or cruises. A worker can relay messages from Cerb to their mobile email application, reply at their leisure, and their messages will be delivered when network access is restored. The feature is also useful when certain workers handle requests through email without logging into the Cerb interface (e.g. temps, interns, consultants). A human dispatcher can route messages to these workers on an as-needed basis. [Mail/Relay/Activity Log] An Activity Log entry is now created when messages are relayed to an email account by a worker or Virtual Attendant. This notifies other workers that work may be taking place on the ticket from outside of Cerb, and it also serves to keep a better history of the actions of Virtual Attendants. Previously, there was no record on a ticket that it had been relayed unless a comment was explicitly created to say so. [Mail/Activity Log/Preferences] Workers can now enable the display of Activity Log entries on the ticket timeline. This displays a mix of messages, comments, and activity log data in a single place without having to click into the 'Activity Log' tab. The option is located on the Settings page that is accessed from the menu displayed when a worker clicks on their name in the top right of the UI. Consecutive log entries will be combined in a single block, and those that take place with a similar timestamp are grouped together for readability. This option is also compatible with the 'Read all' option where the timeline is displayed expanded in 'oldest first' order rather than 'newest first'. [Devblocks/Platform/jQuery] Updated jQuery dependency from 1.7.2 to 1.10.2 [Devblocks/Platform/jQuery] Updated jQuery UI dependency from 1.9.1 to 1.10.3 [Devblocks/Platform/Dependencies] Added the jQuery Caret plugin for getting and setting the cursor position in textareas. [Devblocks/Platform/jqPlot] Updated jqplot dependency from 1.0.0b2 to 1.0.8 [Devblocks/Platform/qTip] Updated qTip dependency from 1.0.0rc3 to 2.2.0 [Tour/Usability] Updated and expanded the tour so it provides tips and callouts about the main functionality within Cerb. This should help onboard new workers faster. [Tour/Usability] When clicking on a new 'Point of Interest' callout in tour mode, the previously viewed callout will now be closed automatically. [CHD-786] [Mail/HTML] Workers can now send HTML formatted email replies from Cerb. This feature was a long time coming, as we wanted to implement it in a way that was consistent with Cerb's spirit of productivity and automation, which are heavily based on the efficient processing of large amounts of text. To facilitate this, the feature is divided into two parts: HTML templates and Markdown syntax. Multiple HTML templates (e.g. letterhead and stylesheets) can be created from the Setup->Mail->HTML page. Each group and bucket can be configured to use a specific template by default, which determines the style of their HTML replies. Templates can contain any valid HTML, although most mail apps will sanitize the messages (e.g. removing Javascript). Rather than introducing a visual HTML editor which obscures the underlying text, workers can reply with Markdown syntax to bold, italicize, link, embed images, quote, and to add headings, code blocks, and lists. Markdown is a markup language that was designed for use in email to preserve human readability. This "progressive enhancement" approach provides workers with greater flexibility in their replies without compromising efficiency. Outgoing messages in the Cerb history are still displayed as plaintext. Additionally, Virtual Attendant behaviors are capable of working with Markdown syntax much easier than with HTML directly. [CHD-786] [Mail/Reply/HTML] When workers are replying to mail, an editor is now provided to assist with using Markdown formatting. By default the editor starts in plaintext mode, which behaves the same as Cerb always has (i.e. only sending text responses). In this case the toolbar only displays a single button that toggles rich text functionality. Once that button is clicked, the toolbar expands to provide: bold, italics, unordered and ordered lists, embedded images, external images, links, quotes, code blocks, and a preview option. The preview option displays the HTML version of the current reply in a popup window. It will also use the HTML template of the current group and bucket, if available. The preview window can be left open while responding, and it will update its contents every time the RETURN key is pressed. [Storage/Files] Attachments can now be requested by their SHA-1 hash in a URL, in addition to the globally unique ID of any of their links. This makes it possible to refer to attachments that have been uploaded but not linked to anything yet. For instance, this is useful for implementing embedded images in mail and knowledgebase articles; those images would need to be previewed before the record (and link) is created. [Platform/Usability] Popups will now open in the middle of the browser window by default. They previously opened in the top center, which made smaller popups less noticeable. [Plugins/Platform] When a file is uploaded using the file chooser popup, it now provides the caller with a SHA-1 hash and internal URL, as well as the entire response object (name, size, mime type, etc). This should make the file chooser reusable in more situations; for instance, in providing embedded image functionality. [Mail/Reply/HTML] Workers can now send inline images in their replies. A new 'Upload an Image' button is available in the toolbar when replying to mail with HTML formatting. This button displays the file chooser popup, which automatically generates the syntax for an embedded image once saved. The image is visible in the preview popup, and once the message is sent Cerb will automatically convert it to an embedded image for the recipient (i.e. replacing the link to Cerb with a self-contained MIME part). [Mail/Search/Usability] When using Search->Ticket, the results will no longer be restricted to only a worker's own groups. The group filter is now added by default but it can be removed from the editable filters. Previously it was a required filter that couldn't be removed. This has been requested several times by organizations that want workers to see the "Access denied" listing for a ticket when searching by mask or participant, instead of having zero results shown (which leads people to believe they're doing the search wrong). [Platform/Popups/Usability] Popups will now float over the same position in the browser even when the rest of the page scrolls. Previously, popups could be scrolled off the page and become lost. [Mail/Peek/Usability] When viewing the peek popup for a ticket, its group and bucket are now shown as two linked dropdowns. Changing the group in the first dropdown will change the list of available buckets in the second one. Previously, the group and bucket were only visible by clicking the dropdown and looking for the (*) marker, which was cumbersome. Additionally, the previous dropdown also combined groups and buckets into a single long list which was difficult to use. [Mail/Compose/Usability] When composing mail, the 'From:' option now displays the group and bucket in two linked dropdowns instead of one big list. [Mail/Compose/HTML] Added HTML message support to the compose popup. The preview feature will display the template for the currently selected group and bucket. [Storage/Files] The '/files' page now provides cache control headers for the content it serves. This allows a worker's browser to cache certain resources (like images) after their first retrieval so they aren't requested from the server again. This is useful now that content like embedded images in HTML messages are displayed through this page. [Platform/Dependencies] Updated HTMLPurifier library from 4.5 to 4.6. [Platform/Devblocks/Dependencies] Added the 'CSS to Inline Styles' library by Tijs Verkoyen. This provides an easy to way to convert CSS STYLE blocks to inline styles when sending HTML email, as many email readers filter out HEAD, BODY, and STYLE. [Mail/Reply/HTML/Preferences] Added a worker preference for "Always switch to HTML mode" while replying to mail. [Platform/Plugins/Popups] Plugins that use genericAjaxPopup() can now specify the 'target' option using a jQuery Position object; e.g. { my: 'top left', at: 'bottom right', of: 'selector' }. [CHD-3622] [Explore/Usability] Improved the way the last accessed time is recorded for explorer sets. Previously the access timestamp only updated 20% of the time in order to reduce extraneous UPDATE database queries when someone quickly paged through the list. Now, instead, the access time is updated any time at least 30 seconds has elapsed since the last recorded access time. Additionally, the access time is now always recorded when the set is created. The prior inefficiency rarely caused problems, but it would manifest if an explorer set was created shortly before the maintenance scheduled job ran, or if the maintenance job was set to run very often instead of nightly. This could result in explorer sets being purged while they were still in use. [Mail/HTML/Broadcast] Implemented HTML message support in ticket worklist broadcasts from bulk update. [Mail/HTML/Broadcast] Implemented HTML message support in address worklist broadcasts from bulk update. [Mail/HTML/Broadcast] Implemented HTML message support in opportunity worklist broadcasts from bulk update. [Mail/Usability/HTML] When generating a plaintext part from an HTML-only message, Cerb now attempts to preserve whitespace in pre-formatted (PRE) and CODE blocks. [Mail/Usability/HTML] When generating a plaintext part from an HTML-only message, Cerb now attempts to prefix nested BLOCKQUOTE content with '>'. [Mail/HTML/Usability] When sending HTML mail, Cerb will generate a new plaintext part that cleans up Markdown formatting. This plaintext part is sent along with the HTML message (e.g. for mobile and automated mail readers), and it's also what's stored in the conversation history. [REVERT] [Platform/Popups/Usability] Popups no longer open up in the center of the browser window by default. This was causing problems on popups that grow, since they could expand beyond the bottom of the window and constantly require being dragged into place. Popups now display at the top so they have the full browser height to grow before they show a scrollbar. When a popup is opened a position can be defined now, so popups that don't grow (like the file chooser) may still elect to show up in the center of the page. [Setup/Mail/Reply-To/HTML] Reply-to addresses can now specify a default HTML template for outgoing mail. The template set on the default reply-to address is the global default. When a group doesn't have its own HTML template it will default to the one on its reply-to address. [Mail/HTML/Usability] The upgrade script for 6.6 now automatically creates a default HTML template and associates it with the default reply-to address. This allows HTML replies to "just work" without any manual configuration. [Virtual Attendants/Mail/HTML] The 'Send mail' action in Virtual Attendant behaviors now supports sending messages using Markdown and HTML templates. [Virtual Attendants/Mail/HTML] The 'Send mail to recipients' action in Virtual Attendant behaviors now supports sending messages using Markdown and HTML templates. [Mail/HTML/Reply] Workers can now optionally choose a specific HTML template when replying. If nothing is selected, the current group/bucket's default is used. [Mail/HTML/Compose] Workers can now optionally choose a specific HTML template when composing a new message. If nothing is selected, the given group/bucket's default is used. [Mail/HTML/Broadcast/Tickets] Workers can now optionally choose a specific HTML template when broadcasting from ticket worklists. If nothing is selected, the default for the group/bucket is used instead. [Mail/HTML/Broadcast/Addresses] Workers can now optionally choose a specific HTML template when broadcasting from email address worklists. If nothing is selected, the default for the given group/bucket is used instead. [Mail/HTML/Broadcast/CRM/Opps] Workers can now optionally choose a specific HTML template when broadcasting from opportunity worklists. If nothing is selected, the default for the group/bucket is used instead. [Mail/Reply/Usability] When replying to a message, the "Would you like to move this conversation?" option now shows the groups and buckets in two linked dropdowns. Changing the group in the first list automatically displays the available buckets in the second list. Previously, the current group and bucket was displayed as a default "No, leave it in the [name] bucket of [group]" option, and moving the conversation required using a long combined list of all the buckets which was difficult to navigate. The new way is also more efficient when using keyboard shortcuts. [Performance/Worklists/Tickets] When using the "Is in groups of [worker]" filter on ticket worklists, the database query is now optimized if the worker is a member of every group. Previously, this inefficiently checked the group_id on every result anyway. [Support Center/Contact/Usability] The ability to upload multiple files in the Support Center now uses a browser's HTML5 support. [Support Center/Contact/Usability] The file type is now set properly on attachments uploaded through the Support Center. Previously, all uploaded files were being saved as 'application/octet-stream', which forced attachments to download even if they could be displayed in the browser (e.g. images, text files). [Web-API/Contexts] Implemented 'GET /rest/contexts/list.json' in the Web-API for retrieving a list of context IDs and names. The context IDs are used for making various kinds of requests (e.g. attachment links, comments, record links) and there wasn't a comprehensive list available anywhere. This API request will also include any contexts that were added through plugins, which the standard documentation wouldn't include. [CHD-3562] [Web-API/Contexts/Links] Implemented 'POST /rest/contexts/link.json' in the Web-API for adding any number of links to a record. See: http://cerbweb.com/book/latest/developer_guide/rest_api/contexts.html [CHD-3562] [Web-API/Contexts/Links] Implemented 'POST /rest/contexts/unlink.json' in the Web-API for removing any number of links from a record. See: http://cerbweb.com/book/latest/developer_guide/rest_api/contexts.html [Setup/Groups] Fixed an issue when deleting a group where group-owned Virtual Attendants and custom fieldsets may not be cleaned up properly. [Less]

Release notes for Cerb 6.6 Cerb (6.6) is a major functionality update released on January 31, 2014. It contains over 70 new features and usability tweaks from community feedback. See: http://wiki.cerbweb.com/6.6 Core [Platform/Custom Fields] ... [More] Implemented configurable parameters for custom fields. The various custom fields types can now provide additional options to modify how they work. Picklist and multi-checkbox fields had a hardcoded parameter for their option list which has been converted to the new format. [CHD-3582] [Custom Fields/Owners] Added a 'Send worker notifications' option to worker-based custom fields. This makes it easy to add an 'owner' field to any kind of record. More advanced workflows are also possible; for example, having two owner fields for a 'salesperson' and 'sales support'. Owners receive watcher notifications about record activity for the duration of their assignment. They don't need to explicitly watch or unwatch records. [Custom Fields/Links] Implemented a new 'Record Link' custom field type. This is similar to how the Links tab works, except the custom field is formalized and the fields of the linked record will be shown in Virtual Attendant behaviors, snippets, etc. For instance, a time tracking entry could have a custom fieldset with fields for an organization and a domain. Time tracking records could then be filtered based on those links, and the related record information could be used in Virtual Attendants, worklists, and the API. [Setup/Storage/Attachments] Fixed an issue where an attachment worklist with multiple links pointing to the same file resulted in missing rows. [Setup/Attachments/Worklists] Attachment worklists now display the storage profile column as a name rather than an ID. [CHD-3377] [Setup/Attachments/Worklists] Attachment worklists once again provide links to the records containing the files (i.e. the 'Context' column). Previously, a link was provided to the ticket but workers had to dig for the message containing the file. Now, when clicking on the permalink for an email message attachment that specific message will be expanded and focused. [CHD-2971] [Mail/Reply/Usability/Keyboard] When using the keyboard to navigate the ticket reply form, using the attachments upload popup previously broke the TAB flow through the form. Now the focus is returned to the attachments (+) button after files have been uploaded. [Storage/Platform/Optimization/Performance] Cerb now automatically detects when a new storage object duplicates an existing one, and an additional link will be created to the same underlying file. A file is only removed when there are zero links pointing at it (e.g. message or comment attachments). Duplicate attachment detection is implemented for incoming mail, worker replies through the UI, the worker proxy, messages imported from an ImpEx export, and files uploaded through the API. This should save a considerable amount of storage space in environments where the same attachments (e.g. PDFs, forms, ebooks) are sent by workers on a regular basis. Similarly, when the same senders include logos and other content in their signature these can be condensed into a single storage object. Currently, existing storage objects are not hashed since would be very intensive on busy systems. New duplicates will be prevented from version 6.6 onward. However, we'll be providing instructions on how to retroactively hash existing storage objects and remove redundancy (technical note: this involves providing SHA1 hashes in the attachment.storage_sha1hash database column, and redirecting attachment_link records based on that). [Mail/Parser/Performance] The original_message.html attachment is now created more efficiently for new incoming messages. Previously its contents were written to a temporary file that was then read into long-term storage. Now it is saved directly from memory into storage. This improvement also made it easier to modify the contents of the HTML message to improve usability (e.g. rewriting links for inline attachments to Cerb URLs). [Mail/Parser/Usability] When viewing an original_message.html attachment in the browser, inline images will now be properly displayed in the content. For instance, this is useful when customers send screenshots by embedding them in the middle of their reply. Previously, such images were included as attachments on the ticket but they weren't displayed in the HTML contents (technical note: The IMG tags still referenced the MIME content-id header). These IMG links are now rewritten to use Cerb file paths. This process works for all new messages that are received in version 6.6 or later, but it will not fix existing original_message.html attachments. [Devblocks/Platform/Markdown] Added the Parsedown library http://parsedown.org/ to Devblocks. This is a faster, leaner, and more modern replacement for the existing PHP Markdown Extra library we've been using up to this point. [Setup/Storage/S3/Usability] When testing a new S3 storage engine profile, a temporary object will be saved, read, and deleted to verify that the credentials work for all actions. Previously the contents of the bucket were listed, but this didn't verify that content could actually be stored and retrieved. [Setup/Storage/S3/Usability] The S3 storage engine now supports a 'Path prefix' configuration option. Previously all content was saved starting at the top-level of a bucket. Now a single bucket can be shared between multiple instances of Cerb. [Setup/Storage/S3/Security] When configuring the S3 storage engine, the secret key parameter is no longer displayed in the profile popup form. If left blank it will remain unchanged. This is far more secure than transmitting it. [Storage/Performance] The storage service now has the ability to queue multiple delete operations to perform them in a single transaction. This is useful for storage engines where incremental deletes would be very expensive. For instance, S3 would generate an HTTP request per file being deleted, which could potentially be hundreds or thousands. Waiting for that many HTTP requests during cron.maint is undesirable. It is now possible for hundreds of objects to be deleted in a single request. [Setup/Storage/S3/Usability] Implemented batch delete functionality in the S3 storage engine. Up to 500 objects can be deleted from S3 in a single HTTP request. [CHD-2711] [Mail/Relay/Comments] When a worker is replying to a relayed message from an external email account, the #comment tag will now allow comments to span multiple lines. The comment will be terminated when it encounters a blank line or a line starting with another #tag. This was necessary because some email applications, like Gmail, force text/plain content to hard wrap on spaces before 76 characters, and quoted-printable encoding isn't used to preserve the original wrapping. The comment will still appear as a single paragraph in Cerb. [CHD-2711] [Mail/Relay/Comments] A new #start comment tag is available to workers that respond to relayed messages through an external email account. This allows multiple line comments where linefeeds will be preserved. This is especially useful for chunks of formatted text like code fragments and patches. A multiple line comment is terminated with an #end tag on its own line. If a worker's mail application adds hard linefeeds to their comment content, those will be preserved in the comment created by Cerb. [Mail/Parser/Localization] Cerb will now use the iconv PHP extension if it is loaded. This is capable of converting a wider range of encodings than mbstring. For instance, TIS-620 (Thai) and WIN-1258 (Vietnamese) were not properly converted to Unicode by Cerb when only using mbstring. [Mail/Relay] Workers can now manually relay specific messages to external email accounts without having to set up a Virtual Attendant behavior first. The reply menu on each message contains a new 'Relay to worker email' option. Selecting the relay option displays a popup where worker email accounts can be filtered and selected. This feature is particularly useful for situations where workers may need to reply while offline, such as during transoceanic flights or cruises. A worker can relay messages from Cerb to their mobile email application, reply at their leisure, and their messages will be delivered when network access is restored. The feature is also useful when certain workers handle requests through email without logging into the Cerb interface (e.g. temps, interns, consultants). A human dispatcher can route messages to these workers on an as-needed basis. [Mail/Relay/Activity Log] An Activity Log entry is now created when messages are relayed to an email account by a worker or Virtual Attendant. This notifies other workers that work may be taking place on the ticket from outside of Cerb, and it also serves to keep a better history of the actions of Virtual Attendants. Previously, there was no record on a ticket that it had been relayed unless a comment was explicitly created to say so. [Mail/Activity Log/Preferences] Workers can now enable the display of Activity Log entries on the ticket timeline. This displays a mix of messages, comments, and activity log data in a single place without having to click into the 'Activity Log' tab. The option is located on the Settings page that is accessed from the menu displayed when a worker clicks on their name in the top right of the UI. Consecutive log entries will be combined in a single block, and those that take place with a similar timestamp are grouped together for readability. This option is also compatible with the 'Read all' option where the timeline is displayed expanded in 'oldest first' order rather than 'newest first'. [Devblocks/Platform/jQuery] Updated jQuery dependency from 1.7.2 to 1.10.2 [Devblocks/Platform/jQuery] Updated jQuery UI dependency from 1.9.1 to 1.10.3 [Devblocks/Platform/Dependencies] Added the jQuery Caret plugin for getting and setting the cursor position in textareas. [Devblocks/Platform/jqPlot] Updated jqplot dependency from 1.0.0b2 to 1.0.8 [Devblocks/Platform/qTip] Updated qTip dependency from 1.0.0rc3 to 2.2.0 [Tour/Usability] Updated and expanded the tour so it provides tips and callouts about the main functionality within Cerb. This should help onboard new workers faster. [Tour/Usability] When clicking on a new 'Point of Interest' callout in tour mode, the previously viewed callout will now be closed automatically. [CHD-786] [Mail/HTML] Workers can now send HTML formatted email replies from Cerb. This feature was a long time coming, as we wanted to implement it in a way that was consistent with Cerb's spirit of productivity and automation, which are heavily based on the efficient processing of large amounts of text. To facilitate this, the feature is divided into two parts: HTML templates and Markdown syntax. Multiple HTML templates (e.g. letterhead and stylesheets) can be created from the Setup->Mail->HTML page. Each group and bucket can be configured to use a specific template by default, which determines the style of their HTML replies. Templates can contain any valid HTML, although most mail apps will sanitize the messages (e.g. removing Javascript). Rather than introducing a visual HTML editor which obscures the underlying text, workers can reply with Markdown syntax to bold, italicize, link, embed images, quote, and to add headings, code blocks, and lists. Markdown is a markup language that was designed for use in email to preserve human readability. This "progressive enhancement" approach provides workers with greater flexibility in their replies without compromising efficiency. Outgoing messages in the Cerb history are still displayed as plaintext. Additionally, Virtual Attendant behaviors are capable of working with Markdown syntax much easier than with HTML directly. [CHD-786] [Mail/Reply/HTML] When workers are replying to mail, an editor is now provided to assist with using Markdown formatting. By default the editor starts in plaintext mode, which behaves the same as Cerb always has (i.e. only sending text responses). In this case the toolbar only displays a single button that toggles rich text functionality. Once that button is clicked, the toolbar expands to provide: bold, italics, unordered and ordered lists, embedded images, external images, links, quotes, code blocks, and a preview option. The preview option displays the HTML version of the current reply in a popup window. It will also use the HTML template of the current group and bucket, if available. The preview window can be left open while responding, and it will update its contents every time the RETURN key is pressed. [Storage/Files] Attachments can now be requested by their SHA-1 hash in a URL, in addition to the globally unique ID of any of their links. This makes it possible to refer to attachments that have been uploaded but not linked to anything yet. For instance, this is useful for implementing embedded images in mail and knowledgebase articles; those images would need to be previewed before the record (and link) is created. [Platform/Usability] Popups will now open in the middle of the browser window by default. They previously opened in the top center, which made smaller popups less noticeable. [Plugins/Platform] When a file is uploaded using the file chooser popup, it now provides the caller with a SHA-1 hash and internal URL, as well as the entire response object (name, size, mime type, etc). This should make the file chooser reusable in more situations; for instance, in providing embedded image functionality. [Mail/Reply/HTML] Workers can now send inline images in their replies. A new 'Upload an Image' button is available in the toolbar when replying to mail with HTML formatting. This button displays the file chooser popup, which automatically generates the syntax for an embedded image once saved. The image is visible in the preview popup, and once the message is sent Cerb will automatically convert it to an embedded image for the recipient (i.e. replacing the link to Cerb with a self-contained MIME part). [Mail/Search/Usability] When using Search->Ticket, the results will no longer be restricted to only a worker's own groups. The group filter is now added by default but it can be removed from the editable filters. Previously it was a required filter that couldn't be removed. This has been requested several times by organizations that want workers to see the "Access denied" listing for a ticket when searching by mask or participant, instead of having zero results shown (which leads people to believe they're doing the search wrong). [Platform/Popups/Usability] Popups will now float over the same position in the browser even when the rest of the page scrolls. Previously, popups could be scrolled off the page and become lost. [Mail/Peek/Usability] When viewing the peek popup for a ticket, its group and bucket are now shown as two linked dropdowns. Changing the group in the first dropdown will change the list of available buckets in the second one. Previously, the group and bucket were only visible by clicking the dropdown and looking for the (*) marker, which was cumbersome. Additionally, the previous dropdown also combined groups and buckets into a single long list which was difficult to use. [Mail/Compose/Usability] When composing mail, the 'From:' option now displays the group and bucket in two linked dropdowns instead of one big list. [Mail/Compose/HTML] Added HTML message support to the compose popup. The preview feature will display the template for the currently selected group and bucket. [Storage/Files] The '/files' page now provides cache control headers for the content it serves. This allows a worker's browser to cache certain resources (like images) after their first retrieval so they aren't requested from the server again. This is useful now that content like embedded images in HTML messages are displayed through this page. [Platform/Dependencies] Updated HTMLPurifier library from 4.5 to 4.6. [Platform/Devblocks/Dependencies] Added the 'CSS to Inline Styles' library by Tijs Verkoyen. This provides an easy to way to convert CSS STYLE blocks to inline styles when sending HTML email, as many email readers filter out HEAD, BODY, and STYLE. [Mail/Reply/HTML/Preferences] Added a worker preference for "Always switch to HTML mode" while replying to mail. [Platform/Plugins/Popups] Plugins that use genericAjaxPopup() can now specify the 'target' option using a jQuery Position object; e.g. { my: 'top left', at: 'bottom right', of: 'selector' }. [CHD-3622] [Explore/Usability] Improved the way the last accessed time is recorded for explorer sets. Previously the access timestamp only updated 20% of the time in order to reduce extraneous UPDATE database queries when someone quickly paged through the list. Now, instead, the access time is updated any time at least 30 seconds has elapsed since the last recorded access time. Additionally, the access time is now always recorded when the set is created. The prior inefficiency rarely caused problems, but it would manifest if an explorer set was created shortly before the maintenance scheduled job ran, or if the maintenance job was set to run very often instead of nightly. This could result in explorer sets being purged while they were still in use. [Mail/HTML/Broadcast] Implemented HTML message support in ticket worklist broadcasts from bulk update. [Mail/HTML/Broadcast] Implemented HTML message support in address worklist broadcasts from bulk update. [Mail/HTML/Broadcast] Implemented HTML message support in opportunity worklist broadcasts from bulk update. [Mail/Usability/HTML] When generating a plaintext part from an HTML-only message, Cerb now attempts to preserve whitespace in pre-formatted (PRE) and CODE blocks. [Mail/Usability/HTML] When generating a plaintext part from an HTML-only message, Cerb now attempts to prefix nested BLOCKQUOTE content with '>'. [Mail/HTML/Usability] When sending HTML mail, Cerb will generate a new plaintext part that cleans up Markdown formatting. This plaintext part is sent along with the HTML message (e.g. for mobile and automated mail readers), and it's also what's stored in the conversation history. [REVERT] [Platform/Popups/Usability] Popups no longer open up in the center of the browser window by default. This was causing problems on popups that grow, since they could expand beyond the bottom of the window and constantly require being dragged into place. Popups now display at the top so they have the full browser height to grow before they show a scrollbar. When a popup is opened a position can be defined now, so popups that don't grow (like the file chooser) may still elect to show up in the center of the page. [Setup/Mail/Reply-To/HTML] Reply-to addresses can now specify a default HTML template for outgoing mail. The template set on the default reply-to address is the global default. When a group doesn't have its own HTML template it will default to the one on its reply-to address. [Mail/HTML/Usability] The upgrade script for 6.6 now automatically creates a default HTML template and associates it with the default reply-to address. This allows HTML replies to "just work" without any manual configuration. [Virtual Attendants/Mail/HTML] The 'Send mail' action in Virtual Attendant behaviors now supports sending messages using Markdown and HTML templates. [Virtual Attendants/Mail/HTML] The 'Send mail to recipients' action in Virtual Attendant behaviors now supports sending messages using Markdown and HTML templates. [Mail/HTML/Reply] Workers can now optionally choose a specific HTML template when replying. If nothing is selected, the current group/bucket's default is used. [Mail/HTML/Compose] Workers can now optionally choose a specific HTML template when composing a new message. If nothing is selected, the given group/bucket's default is used. [Mail/HTML/Broadcast/Tickets] Workers can now optionally choose a specific HTML template when broadcasting from ticket worklists. If nothing is selected, the default for the group/bucket is used instead. [Mail/HTML/Broadcast/Addresses] Workers can now optionally choose a specific HTML template when broadcasting from email address worklists. If nothing is selected, the default for the given group/bucket is used instead. [Mail/HTML/Broadcast/CRM/Opps] Workers can now optionally choose a specific HTML template when broadcasting from opportunity worklists. If nothing is selected, the default for the group/bucket is used instead. [Mail/Reply/Usability] When replying to a message, the "Would you like to move this conversation?" option now shows the groups and buckets in two linked dropdowns. Changing the group in the first list automatically displays the available buckets in the second list. Previously, the current group and bucket was displayed as a default "No, leave it in the [name] bucket of [group]" option, and moving the conversation required using a long combined list of all the buckets which was difficult to navigate. The new way is also more efficient when using keyboard shortcuts. [Performance/Worklists/Tickets] When using the "Is in groups of [worker]" filter on ticket worklists, the database query is now optimized if the worker is a member of every group. Previously, this inefficiently checked the group_id on every result anyway. [Support Center/Contact/Usability] The ability to upload multiple files in the Support Center now uses a browser's HTML5 support. [Support Center/Contact/Usability] The file type is now set properly on attachments uploaded through the Support Center. Previously, all uploaded files were being saved as 'application/octet-stream', which forced attachments to download even if they could be displayed in the browser (e.g. images, text files). [Web-API/Contexts] Implemented 'GET /rest/contexts/list.json' in the Web-API for retrieving a list of context IDs and names. The context IDs are used for making various kinds of requests (e.g. attachment links, comments, record links) and there wasn't a comprehensive list available anywhere. This API request will also include any contexts that were added through plugins, which the standard documentation wouldn't include. [CHD-3562] [Web-API/Contexts/Links] Implemented 'POST /rest/contexts/link.json' in the Web-API for adding any number of links to a record. See: http://cerbweb.com/book/latest/developer_guide/rest_api/contexts.html [CHD-3562] [Web-API/Contexts/Links] Implemented 'POST /rest/contexts/unlink.json' in the Web-API for removing any number of links from a record. See: http://cerbweb.com/book/latest/developer_guide/rest_api/contexts.html [Setup/Groups] Fixed an issue when deleting a group where group-owned Virtual Attendants and custom fieldsets may not be cleaned up properly. [Less]

Introduction Cerb is a complex project with a 12 year history and more than 300,000 lines of source code. During this time there has been at least one known issue where file attachments on deleted records were not properly cleaned up on the disk. ... [More] This article provides instructions on how to compare the contents of the database and the filesystem, as well as instructions on how to clean up any inconsistencies that are found. Instructions Make a backup! The instructions in this article require the use of several console commands on your server. If these commands are entered improperly by accident then you may experience unintended data loss. Please make a current backup of your database and filesystem before proceeding. Creating a list of files on the disk Change directory to the attachments directory of the storage filesystem: cd /path/to/cerb5/storage/attachments Create a sorted list of all the files on the disk: find * -type f | sort > files.disk Creating a list of files in the database From the ./storage/attachments directory, run the following command: mysql -u -p -BN -e \ "SELECT storage_key FROM attachment \ WHERE storage_extension = 'devblocks.storage.engine.disk' \ ORDER BY storage_key" \ | sort > files.db You will need to replace , , and with the proper values for your environment. If you are unsure of these values, consult your framework.config.php file. Checking file storage integrity Count files only on disk and not in the database comm -23 files.disk files.db | wc -l Count files only in the database and not on disk comm -13 files.disk files.db | wc -l Count where a file is both on disk and in the database comm -12 files.disk files.db | wc -l Cleaning orphaned files Cleaning up orphaned files on disk If you have files that only appear on the disk and are no longer in the database, you can transform the comparison list into a sequence of commands in order to remove or relocate the files. Make sure you are still in the ./storage/attachments directory. To remove these files, iterate over the list and delete each one: for f in `comm -23 files.disk files.db`; do rm -v $f; done; Cleaning up orphaned files in the database If you have files that only appear in the database and are no longer on the disk, you can transform the comparison list into a series of SQL statements to delete the records. Make sure you are still in the ./storage/attachments directory. Iterate over the list of orphaned files to generate a list of SQL statements in order to delete the affected rows: for key in `comm -13 files.disk files.db`; do echo "DELETE FROM attachment WHERE id = `basename ${key}`;" >> deletes.sql; done; You now have a file named deletes.sql with the statements required to clean up the database. The recommended way to execute these statements is to pipe them through the mysql command: mysql --verbose -u -p < deletes.sql Again, you will need to replace , , and with the proper values for your environment. When finished, delete the SQL statements file: rm deletes.sql Finishing up Remove the temporary files you created during this process: rm files.disk files.db [Less]

Introduction Cerb is a complex project with a 12 year history and more than 300,000 lines of source code. During this time there has been at least one known issue where file attachments on deleted records were not properly cleaned up on the disk. ... [More] This article provides instructions on how to compare the contents of the database and the filesystem, as well as instructions on how to clean up any inconsistencies that are found. Instructions Make a backup! The instructions in this article require the use of several console commands on your server. If these commands are entered improperly by accident then you may experience unintended data loss. Please make a current backup of your database and filesystem before proceeding. Creating a list of files on the disk Change directory to the attachments directory of the storage filesystem: cd /path/to/cerb5/storage/attachments Create a sorted list of all the files on the disk: find * -type f | sort > files.disk Creating a list of files in the database From the ./storage/attachments directory, run the following command: mysql -u <user> -p<pass> -BN -e \ "SELECT storage_key FROM attachment \ WHERE storage_extension = 'devblocks.storage.engine.disk' \ ORDER BY storage_key" <database> \ | sort > files.db You will need to replace <user>, <pass>, and <database> with the proper values for your environment. If you are unsure of these values, consult your framework.config.php file. Checking file storage integrity Count files only on disk and not in the database comm -23 files.disk files.db | wc -l Count files only in the database and not on disk comm -13 files.disk files.db | wc -l Count where a file is both on disk and in the database comm -12 files.disk files.db | wc -l Cleaning orphaned files Cleaning up orphaned files on disk If you have files that only appear on the disk and are no longer in the database, you can transform the comparison list into a sequence of commands in order to remove or relocate the files. Make sure you are still in the ./storage/attachments directory. To remove these files, iterate over the list and delete each one: for f in `comm -23 files.disk files.db`; do rm -v $f; done; Cleaning up orphaned files in the database If you have files that only appear in the database and are no longer on the disk, you can transform the comparison list into a series of SQL statements to delete the records. Make sure you are still in the ./storage/attachments directory. Iterate over the list of orphaned files to generate a list of SQL statements in order to delete the affected rows: for key in `comm -13 files.disk files.db`; do echo "DELETE FROM attachment WHERE id = `basename ${key}`;" >> deletes.sql; done; You now have a file named deletes.sql with the statements required to clean up the database. The recommended way to execute these statements is to pipe them through the mysql command: mysql --verbose -u <user> -p<pass> <database> < deletes.sql Again, you will need to replace <user>, <pass>, and <database> with the proper values for your environment. When finished, delete the SQL statements file: rm deletes.sql Finishing up Remove the temporary files you created during this process: rm files.disk files.db [Less]

Introduction Cerb is a complex project with a 12 year history and more than 300,000 lines of source code. During this time there has been at least one known issue where file attachments on deleted records were not properly cleaned up on the disk. ... [More] This article provides instructions on how to compare the contents of the database and the filesystem, as well as instructions on how to clean up any inconsistencies that are found. Instructions Make a backup! The instructions in this article require the use of several console commands on your server. If these commands are entered improperly by accident then you may experience unintended data loss. Please make a current backup of your database and filesystem before proceeding. Creating a list of files on the disk Change directory to the attachments directory of the storage filesystem: cd /path/to/cerb5/storage/attachments Create a sorted list of all the files on the disk: find * -type f | sort > files.disk Creating a list of files in the database From the ./storage/attachments directory, run the following command: mysql -u <user> -p<pass> -BN -e \ "SELECT storage_key FROM attachment \ WHERE storage_extension = 'devblocks.storage.engine.disk' \ ORDER BY storage_key" <database> \ | sort > files.db You will need to replace <user>, <pass>, and <database> with the proper values for your environment. If you are unsure of these values, consult your framework.config.php file. Checking file storage integrity Count files only on disk and not in the database comm -23 files.disk files.db | wc -l Count files only in the database and not on disk comm -13 files.disk files.db | wc -l Count where a file is both on disk and in the database comm -12 files.disk files.db | wc -l Cleaning orphaned files Cleaning up orphaned files on disk If you have files that only appear on the disk and are no longer in the database, you can transform the comparison list into a sequence of commands in order to remove or relocate the files. Make sure you are still in the ./storage/attachments directory. To remove these files, iterate over the list and delete each one: for f in `comm -23 files.disk files.db`; do rm -v $f; done; Cleaning up orphaned files in the database If you have files that only appear in the database and are no longer on the disk, you can transform the comparison list into a series of SQL statements to delete the records. Make sure you are still in the ./storage/attachments directory. Iterate over the list of orphaned files to generate a list of SQL statements in order to delete the affected rows: for key in `comm -13 files.disk files.db`; do echo "DELETE FROM attachment WHERE id = `basename ${key}`;" >> deletes.sql; done; You now have a file named deletes.sql with the statements required to clean up the database. The recommended way to execute these statements is to pipe them through the mysql command: mysql --verbose -u <user> -p<pass> <database> < deletes.sql Again, you will need to replace <user>, <pass>, and <database> with the proper values for your environment. When finished, delete the SQL statements file: rm deletes.sql Finishing up Remove the temporary files you created during this process: rm files.disk files.db [Less]

Introduction Cerb is a complex project with a 12 year history and more than 300,000 lines of source code. During this time there has been at least one known issue where file attachments on deleted records were not properly cleaned up on the disk. ... [More] This article provides instructions on how to compare the contents of the database and the filesystem, as well as instructions on how to clean up any inconsistencies that are found. Instructions Make a backup! The instructions in this article require the use of several console commands on your server. If these commands are entered improperly by accident then you may experience unintended data loss. Please make a current backup of your database and filesystem before proceeding. Creating a list of files on the disk Change directory to the attachments directory of the storage filesystem: cd /path/to/cerb5/storage/attachments Create a sorted list of all the files on the disk: find * -type f | sort > files.disk Creating a list of files in the database From the ./storage/attachments directory, run the following command: mysql -u <user> -p<pass> -BN -e \ "SELECT storage_key FROM attachment \ WHERE storage_extension = 'devblocks.storage.engine.disk' \ ORDER BY storage_key" <database> \ | sort > files.db You will need to replace <user>, <pass>, and <database> with the proper values for your environment. If you are unsure of these values, consult your framework.config.php file. Checking file storage integrity Count files only on disk and not in the database comm -23 files.disk files.db | wc -l Count files only in the database and not on disk comm -13 files.disk files.db | wc -l Count where a file is both on disk and in the database comm -12 files.disk files.db | wc -l Cleaning orphaned files Cleaning up orphaned files on disk If you have files that only appear on the disk and are no longer in the database, you can transform the comparison list into a sequence of commands in order to remove or relocate the files. Make sure you are still in the ./storage/attachments directory. To remove these files, iterate over the list and delete each one: for f in `comm -23 files.disk files.db`; do rm -v $f; done; Cleaning up orphaned files in the database If you have files that only appear in the database and are no longer on the disk, you can transform the comparison list into a series of SQL statements to delete the records. Make sure you are still in the ./storage/attachments directory. Iterate over the list of orphaned files to generate a list of SQL statements in order to delete the affected rows: for key in `comm -13 files.disk files.db`; do echo "DELETE FROM attachment WHERE id = `basename ${key}`;" >> deletes.sql; done; You now have a file named deletes.sql with the statements required to clean up the database. The recommended way to execute these statements is to pipe them through the mysql command: mysql --verbose -u <user> -p<pass> <database> < deletes.sql Again, you will need to replace <user>, <pass>, and <database> with the proper values for your environment. When finished, delete the SQL statements file: rm deletes.sql Finishing up Remove the temporary files you created during this process: rm files.disk files.db [Less]

Introduction The default Support Center uses the Cerberus Helpdesk logo and a fairly plain color scheme. Most administrators will want to change the logo used, and many will also want to change the color scheme to match the corporate / institutional ... [More] colors. This article will show you how to do both of those. Modifying your Support Center with custom templates Click Setup on the right of the global navigation menu. Click the Configure menu and select Community Portals. Click on the Support Center that you would like to edit. Select the Custom Templates tab. Changing the default logo Click the Add Custom Template button. Select the template [cerberusweb.support_center] support_center/header.tpl and click Save Changes. The second line reads, in part: src="{devblocks_url}c=resource&p=cerberusweb.support_center&f=images/_wgm/logo.gif{/devblocks_url}" Replace the contents of the image src property with a link to your corporate / institutional logo, like: src="http://example.com/images/logo.png" Click the Save Changes button. Changing the color scheme Click Add Custom Template button. Select template: [cerberusweb.support_center] support_center/style.css.tpl and click Save Changes. You can edit any of the CSS styles of the support center here, but as an example, we'll change the H1 tag from blue to green. First, find the section that says: H1 { font-size:20px; font-weight:bold; color: rgb(8,90,173); margin-top:0px; margin-bottom:3px; } and then edit the color to: color: rgb(0,120,0); Click the Save Changes button. Test it! Browse to your Support Center. You should now see your new logo and color scheme. [Less]