1-23 start/2007-02-28/1

Don't get overexcited because I've finished the tests I wanted to do. The extra tests I've made include paging and the computation of a value with some cuts like João suggested. I've also redone B test because I've introduced a few more optimizations in the DB level (some extra indexes).

As you can see the test from B was reduced to less than 1/3 of the original value.

The D test was ommited. In it I tryed to measure the time it takes to load a single record. It proved meaningless. It doesn't take long (around 0.30) and the "real" cases hardly ever loaded a single record. We normally loaded an entire 1-n relation.

The E test was a query like "SELECT SUM(...) … WHERE status = <value> and date BETWEEN <begin> and <end>". The in memory computation required us to process each Order and filter the order by status and date and then compute the sum for all OrderItems in the orders that weren't filtered. As you can see, DB clearly beated the in-memory processing. João was righ when considering that database cut's based on indexes are very efficient compared to in-memory graph traveling. As an additional note I've found that, in F, most of the time was spent comparing dates (creating a calendar, seting the date and using compareTo()).

The last group of tests paging. In-memory paging, that is H, includes creating a new List from the list of persisted objects, sorting that list and geting the subpage (subList(start, start+offset)).

So, in-memory computation can beat the db in some situations. We lack the kind of infrastructure the database has when regarding query optimization, indexes, cuts, etc. That infrastructure could be partially implemented in a framework to optimize the selection of objects in memory and increase the cases were in-memory beasts communicating with the database. Nevertheless there is a huge effort in the DB backend and I'm not sure how much of that effort would be needed to replicate in such framework. We also have the scalability isssue ...

Guess this didn't gave us THE answer to our problems but at least we have some "pretty" graphics. Uh! Shinnny!!!.