Consolidation tasks may consume all CPUs and defer other consolidations

[valeriansaliou]

Issue observed at Crisp's scale (https://crisp.chat/); it's been a week that after a smooth sailing, our Sonic instance started consuming A LOT of CPU, on consolidation tasks (first guess).

An investigation of the heaviness / slowdown needs to be proceeded, and the performance issue needs to be fixed.

[valeriansaliou]

That may also be due to RocksDB, need to investigate.

[valeriansaliou]

Targets:

• When pushing new words to FST graph, check if FST is already over limits or not (do nothing if over limits, if push);
• Attempt to optimize FST consolidation algorithm;