DataCleaner

We have just released the newest version of MetaModel, 1.1.5. This release is a minor release which means no API changes, but a few upgrades in terms of performance, flexibility and ease of distribution (full list): The most important upgrade have ... [More] been to CSV performance. We encountered a bug when querying this type of datastore that meant that the whole DataSet was stored in memory while using it. This has undergone quite some refactoring so that it will now stream through memory as expected, thus keeping the door open for very large CSV files. A minor change in the column naming scheme have been implemented for the Excel-based DataContext's. This means that if the first row of a spreadsheet contains only blank fields, we will automatically assign the names "[column 1]", "[column 2]" etc. accordingly. The downloadable zip or tar.gz file will now contain a "MetaModel-1.1.5-all.jar" file, which is an assembled jar file containing the classes of all MetaModel modules (core, csv, jdbc, excel etc.), which should substantially ease deployment of the framework. We hope you enjoy the new release of MetaModel and keep up the good work of providing the valuable feedback that drives development of it. [Less]

We're happy to announce the release of DataCleaner version 1.5.1. This release is a minor release, nevertheless containing a few nice features - especially for the users who are enjoying the exporting features that was introduced in 1.5: An ... [More] additional HTML export format have been added to the built-in export formats (usable when exporting Profiler results in the desktop app and when executing the runjob command-line tool). The export format is now choosable directly in the desktop app. Four new measures where added to the String Analysis profile: avg. chars and max/min/avg white spaces. The new version of DataCleaner is (as always) downloadable for free on the downloads page and feedback from users is also greatly appreciated, ie: Fill out our online user survey, or Post your comments and questions at our discussion forum. We hope that you all enjoy DataCleaner 1.5.1. [Less]

"Finally!" one might say. And this is definately what is going through my head right as I write this news-item. Finally, DataCleaner 1.5 has been released! Once again the effort to bring about the best open source data quality solution is bearing ... [More] fruit. The new release is definately one of the most significant ones in the history of DataCleaner. The overall goal of the release has been to step up from the shadows of the "small tools" pool and mark DataCleaner as an enterprise-ready application for profiling and validating datastores of all kinds - both in scheduled mode, on servers and in an intuitive desktop environment. For those of you with an interest in every little detail about this release, please feel free to review the complete list of changes - for everyone else, here's the recap: Change of license to LGPL. Multi-threaded execution of Profiler and Validator. Command line (batch) execution of DataCleaner tasks. More elaborate status information during profiler and validator execution. New profile: Date mask matcher. New profile: Regex matcher. Load regex from the online RegexSwap repository. Automatic download and install of popular database drivers. More file types supported (.dat, .txt) XML file support improved (.xml) Memory improvements in Time analysis profile. Improved logging when running profiling and validation. Information schema provided for file-based datastores. Lazy-loading of columns in datastore-tree. We hope you enjoy the new DataCleaner 1.5! Now go over and download it right away. [Less]

Things are starting to shape up for the big release of DataCleaner 1.5. We are starting off with a bit of excitement around in the data quality community. Probably the most dedicated online magazine about data quality, data quality pro, have ... [More] launched a series of articles about profiling, validating and comparing data with DataCleaner. So far an introductory tutorial (including a complete and realistic example data-set) and a background article/interview have been published: Learn how to profile and validate data (for free) using DataCleaner Interview with Kasper Sørensen, creator of DataCleaner We hope that you will enjoy the articles and we thank data quality pro for their great interest in our community. [Less]

Today we are announcing the first company, Lund&Bendsen, to officially support DataCleaner and MetaModel on a commercial level. These eobjects.org projects are, as you know, independent projects that are run with the community in mind. But as ... [More] time goes on they grow and for companies to pick them up and start using them in a commercial setting we also welcome third party commercial support to help spread the projects to environments where community-based support is insufficient. Lund&Bendsen is a Danish company with a strong expertise in Java development and training. Their service offerings include training, customization, integration and enhancement of DataCleaner and MetaModel so if your company is considering applying DataCleaner they might be interested in hiring some professionals to aid them in the process. Over time more companies are expected to join in on commercial support for the eobjects.org projects. Keep up to date on the DataCleaner support page and don't hesitate to contact us for any inquiries in this regard either. [Less]

The Technology Evaluation Centers (TEC) have published an interesting, unbiased and independent analysis of the market for Open Source business intelligence products. We are delighted to see that the article features a section about data quality and ... [More] that TEC points at DataCleaner as a competent choise within the open source products: In such situations, where the vendor does not support a specific functionality, organizations can look to complementary open source solutions; the DataCleaner project from eobjects.org, for instance, provides functionality to help profile data and monitor data quality. It also points to a significant advantage with open source applications: the fact that software is developed by the community and for the community makes it much simpler to share innovative solutions quickly and seamlessly. You can read the whole article by Anna Mallikarjunan from TEC by going to their website (user registration is required). [Less]

Another batch of updates, fixes and improvements for the upcoming DataCleaner release is ready. This time it's Release Candidate 2 offering a preview of what's to come in DataCleaner 1.5. DataCleaner download site: ... [More] http://datacleaner.eobjects.org/downloads The main changes since Release Candidate 1 are multithreaded execution, the command line interface (runjob.sh / runjob.cmd), some UI updates and a few bugfixes. Go download the release candidate and use it as an opportunity to influence the development process by posting your comments on the DataCleaner forum. [Less]

After working hard for a couple of days to implement substantial new features regarding integration of eobjects services and automatic download and install of popular database drivers, a new release candidate of DataCleaner is ready! DataCleaner ... [More] download site: http://datacleaner.eobjects.org/downloads We hope that a lot of people will use the release candidate and provide feedback for further development towards the 1.5 final release. [Less]

I've spent the last couple of days implementing a couple of cool enhancements to the DataCleaner desktop-application: Automatic download and install of popular database drivers. Followed along with template connection strings in the "Open database" ... [More] dialog. This will hopefully make it much easier for less experienced users to set up a connection to their database of choice. Direct integration with the new RegexSwap system so that the regexes that you post online will be accessible from within the desktop-application. Screenshots have been posted to the media page. Wait for DataCleaner 1.5 for these features or build it yourself to check them out now. [Less]

A new release of MetaModel is ready for download. The new version, 1.1.4, is a bug-fix release with a critical issue for PostgreSQL databases fixed. Other than that no changes from 1.1.3, so it should be a drop-in replacement update. Enjoy. You ... [More] can download an archived version Or get it using maven: <dependency> <groupId>dk.eobjects.metamodel</groupId> <artifactId>MetaModel-full</artifactId> <version>1.1.4/version> </dependency> [Less]