Tideglass

Tideglass takes a firehose of raw events and turns it into rollups a dashboard can query in milliseconds. The hard part is not the happy path. It is what happens when the firehose surges.

The problem

The old pipeline wrote every event straight to the analytics store. When traffic spiked, the writes backed up, the store slowed, and the slowness propagated upstream until ingestion stalled entirely.

The design

The rewrite treats the database as a resource to protect, not a bucket to fill. A bounded channel between the consumer and the writer turns a surge into controlled slowness rather than collapse.

// bounded channel = built-in backpressure
let (tx, rx) = mpsc::channel::<Event>(10_000);
 
// consumer blocks when the writer falls behind, which slows the read
// from Kafka instead of overwhelming ClickHouse
while let Some(event) = stream.next().await {
    tx.send(event).await?;
}

• Bounded buffering so memory cannot grow without limit.
• Batched inserts sized to the store's sweet spot, not per-event writes.
• Pre-aggregation into rollups so the query side reads small, dense tables.

The outcome

The pipeline now absorbs traffic spikes of roughly ten times baseline without stalling. When the store is briefly slow, ingestion slows with it and recovers, instead of falling over.