Commit Graph

44 Commits (b5768c8bf014727fa746494f899f8d31733f4c29)

Author SHA1 Message Date
Michael Zanetti e52b45c815 fix tests 2021-01-04 18:02:26 +01:00
Michael Zanetti ae38e185b6 more work... not really working yet 2021-01-02 14:33:15 +01:00
Jenkins nymea dc536db74e Merge PR #315: Use bindValue for composing query strings for the log DB 2020-08-14 13:09:34 +02:00
Michael Zanetti 16f80aa24c Fix time filter 2020-07-14 11:23:05 +02:00
Michael Zanetti a4beed9aee Properly store stringlists 2020-07-10 23:15:05 +02:00
Michael Zanetti 98e5176393 Use bindValue for composing query strings for the log DB
Event/action param values might not be escaped properly otherwise
2020-07-05 12:29:38 +02:00
Michael Zanetti bb30a451b3 New DB schema version
* Store plaintext values in log db again
* Use msecs precision instead of secs
* rename column deviceId to thingId
2020-05-04 11:32:42 +02:00
Michael Zanetti 8208473302 bump api 2020-03-05 16:16:57 +01:00
Michael Zanetti 3e425fc55b Rename devices to integrations/things 2020-03-02 13:40:26 +01:00
Michael Zanetti 8f5536ba1a Update copyright 2020-01-23 20:07:26 +01:00
Jenkins nymea 2d08e7a028 Merge PR #241: Improve log DB housekeeping performance 2020-01-22 23:47:15 +01:00
Michael Zanetti 744f25312b Cleanup as code review suggests 2020-01-15 12:32:32 +01:00
Michael Zanetti 03c8e8d114 Improve log DB housekeeping performance
Getting row count on a db with approx 200000 entries on an RPi takes
about 500ms. To avoid this, this branch keeps better track of entries
in the DB and only queries DB count if we can't calculate it ourselves.
Trimming itself takes some 150ms. To reduce those calls it changes the
threshold of when to trim the DB from a fixed value of 100 to 1% of
maxDBSize.

Last but not least, getLogEntry() calls are now prioritized over
appendLogEntry() calls in order to stay responsive to client apps even
if the DB is overloaded with a huge job queue.

If the job queue grows to over 1000 jobs, logs of the same device/type
will be discared to avoid log flooding.
2020-01-05 23:46:51 +01:00
Michael Zanetti 22bacf09e1 fix build 2019-12-01 01:14:26 +01:00
Michael Zanetti 212ccc8596 remove unneeded processQueue 2019-11-30 23:56:56 +01:00
Michael Zanetti 64a494e44a Fix tests that didn't wait long enough for log entries to be written 2019-11-28 20:47:55 +01:00
Michael Zanetti 619bbf76f3 don't use QSqlQuery at all outside the job queue
On some backends, such as MySQL, a prepare() call will access
the database
2019-11-28 14:20:37 +01:00
Michael Zanetti c7dd26d525 cleanup, fix last tests 2019-11-16 00:44:08 +01:00
Michael Zanetti 73812df269 fix shutdown 2019-11-15 23:54:40 +01:00
Michael Zanetti 0ce99fc127 add some debugs 2019-11-15 23:37:52 +01:00
Michael Zanetti f36eac080f fix duplicate query execution 2019-11-15 23:12:27 +01:00
Michael Zanetti 2361cc0473 Wait for the job queue to finish before shutting down 2019-11-15 18:34:20 +01:00
Michael Zanetti 3eac06de6e Threaded log database
This
a) makes the log db threaded by using QtConcurrent to run queries in a
different thread but still keeps ordering of the queries and always
only allows a single query at a time (QSql is not threadsafe). This fixes
removeDevice calls failing if we take more than $jsonprc_timeout to clean
a deleted device from the DB and keeps nymead responsive during that too.

b) generally improces performance of the system by not requiring operations
(emitting events, changing states) to wait for the sync log db entry to be
made.

c) drops some of the houskeeping code on nymea startup. While this will still
do log db housekeeping if the DB exceeds maxDbSize, it will not run
housekeeping on the DB any more at application startup. Reasoning for this
is that there have been reports of rules/log entries beimg destroyed if a
plugin can't be found at application startup. Given our general direction
of working towards more dynamic plugin loading, this might happen more often
in the future and we sure don't want to destroy rules etc when we just
temporarily miss a plugin.

d) tries to fix issue #226 by rotating the DB not only when it fails to open
initially, but also when it fails to insert new entries.
2019-11-15 18:33:10 +01:00
Michael Zanetti 3cbd01c1d2 drop all pack/unpack methds 2019-11-03 21:24:16 +01:00
Michael Zanetti baaa719b6a Restructuring json done 2019-11-01 20:55:45 +01:00
Michael Zanetti bdd9876a88 Allow registering QMetaObjects on JSON api 2019-11-01 20:55:45 +01:00
Michael Zanetti 5e3bc2acbd Drop JsonTypes class by distributing logic to json handlers
This is required in order to be able to be more flexible in registering
new types/methods.
2019-11-01 20:46:55 +01:00
Michael Zanetti aee554a1ee BrowserItemActions, yay! 2019-07-25 15:56:18 +02:00
Michael Zanetti ff337bf8b6 Make browser item execution work async too 2019-07-25 15:56:18 +02:00
Michael Zanetti a385188d37 More work on the device browser 2019-07-25 15:56:18 +02:00
Michael Zanetti d24abfe3d0 Move devicemanager's implementation out of libnymea into libnymea-core 2019-07-15 22:52:52 +02:00
Michael Zanetti b49dafa7e3 add limits and offset to GetLogEntries 2019-04-08 13:55:15 +02:00
Simon Stürz dcd410457d Update core documentation and fix inline warnings 2019-04-08 13:55:15 +02:00
Michael Zanetti 23316942ce use bindValue for the log value filter 2019-04-08 13:55:15 +02:00
Michael Zanetti 8db7e050db Add support for configuring the used database backend 2019-04-08 13:55:15 +02:00
Michael Zanetti b8c11cc850 improve debug output for the ruleengine 2019-04-08 13:55:15 +02:00
Michael Zanetti 67a535619f fix typos as reported by Krazy2 2019-04-08 13:55:15 +02:00
Michael Zanetti 79cc563eb0 docs guh -> nymea 2019-04-08 13:55:15 +02:00
Michael Zanetti b8a86b095d copyright headers guh -> nymea 2019-04-08 13:55:15 +02:00
Michael Zanetti 36d4f89b24 namespace guhserver -> namespace nymeaserver 2019-04-08 13:55:15 +02:00
Michael Zanetti da9fe61044 guhconfiguration -> nymeaconfiguration 2019-04-08 13:55:15 +02:00
Michael Zanetti 559a8be1f6 guhcore -> nymeacore 2019-04-08 13:55:15 +02:00
Michael Zanetti dca764fafd guhsettings -> nymeasettings 2019-04-08 13:55:15 +02:00
Michael Zanetti 4be14f1904 libguh-core -> libnymea-core 2019-04-08 13:55:15 +02:00