Getting row count on a db with approx 200000 entries on an RPi takes
about 500ms. To avoid this, this branch keeps better track of entries
in the DB and only queries DB count if we can't calculate it ourselves.
Trimming itself takes some 150ms. To reduce those calls it changes the
threshold of when to trim the DB from a fixed value of 100 to 1% of
maxDBSize.
Last but not least, getLogEntry() calls are now prioritized over
appendLogEntry() calls in order to stay responsive to client apps even
if the DB is overloaded with a huge job queue.
If the job queue grows to over 1000 jobs, logs of the same device/type
will be discared to avoid log flooding.
This
a) makes the log db threaded by using QtConcurrent to run queries in a
different thread but still keeps ordering of the queries and always
only allows a single query at a time (QSql is not threadsafe). This fixes
removeDevice calls failing if we take more than $jsonprc_timeout to clean
a deleted device from the DB and keeps nymead responsive during that too.
b) generally improces performance of the system by not requiring operations
(emitting events, changing states) to wait for the sync log db entry to be
made.
c) drops some of the houskeeping code on nymea startup. While this will still
do log db housekeeping if the DB exceeds maxDbSize, it will not run
housekeeping on the DB any more at application startup. Reasoning for this
is that there have been reports of rules/log entries beimg destroyed if a
plugin can't be found at application startup. Given our general direction
of working towards more dynamic plugin loading, this might happen more often
in the future and we sure don't want to destroy rules etc when we just
temporarily miss a plugin.
d) tries to fix issue #226 by rotating the DB not only when it fails to open
initially, but also when it fails to insert new entries.
