We've been using Apache Derby embedded into rich clients we've been delivering to various customers. It is undeniable that it strength reside in being able to power any Java application, being it rich client, web application or others completely hidden from the user / installer.
It has been deployed together with a JPA implementation, fully enabling object oriented programming principles to developers. We've used it to distribute the persistence of server side entities to rich clients, enabling users to work offline and then resynchronize with the server as soon as a connection is again available.
Apache Derby has never let us down, providing incredible performances, especially embedded! Weird behavior was encountered due to JPA implementations, not Derby itself. To me it is just the proof of maturity level it has gained over the years. A level that widely used, but also younger framework haven't yet reached!