Observability

Brim Internals

Deep visibility into the semantic search pipeline execution

Pipeline Architecture

BRIM SEMANTIC SEARCH PIPELINE

EMBEDDING GENERATION

1. Text Extraction Model::toEmbeddableText()
2. Text Chunking ~0 chunks
3. Embedding (Ollama) 0ms avg
4. Storage (pgvector) 0ms avg
Total 0ms avg

SEMANTIC SEARCH

1. Query Embedding 736.56ms avg
2. Vector Search (cosine) 5.49ms avg
3. Model Hydration 2.86ms avg
4. Results avg 10 @ 0.642
Total 744.91ms avg

Embedding Generation

0
Total Generated
0ms
Avg Duration
Embedding Time (Ollama) 0ms
Storage Time (pgvector) 0ms
Total Chunks Created 0
Min / Max Duration 0ms / 0ms

Semantic Search

11
Total Searches
744.91ms
Avg Duration
Query Embedding Time 736.56ms
Vector Search Time 5.49ms
Model Hydration Time 2.86ms
Avg Results / Top Score 10 / 0.642
Min / Max Duration 125.35ms / 840ms

Pipeline Timing Breakdown

Embedding Generation

Text Chunking ~5ms
chunk
Ollama Embedding 0ms
embed
pgvector Insert 0ms
store

Semantic Search

Query Embedding 736.56ms
embed
Vector Search (cosine) 5.49ms
search
Model Hydration 2.86ms
hydrate

Event Stream

Time Event Duration Details
09:27:07 search.completed 125.4ms Concept | 10 results | top: 0.628 | embed: 117.3ms | search: 5.3ms | hydrate: 2.8ms
09:27:07 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 23 chars
09:26:40 search.completed 823.8ms Concept | 10 results | top: 0.628 | embed: 815.3ms | search: 5.6ms | hydrate: 2.9ms
09:26:39 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
08:37:39 search.completed 821.6ms Concept | 10 results | top: 0.617 | embed: 813.5ms | search: 5.3ms | hydrate: 2.8ms
08:37:39 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 28 chars
02:24:31 search.completed 793.1ms Concept | 10 results | top: 0.691 | embed: 785.1ms | search: 5.4ms | hydrate: 2.6ms
02:24:30 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 32 chars
22:24:31 search.completed 812.8ms Concept | 10 results | top: 0.685 | embed: 803.6ms | search: 6.1ms | hydrate: 3.1ms
22:24:30 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 14 chars
20:00:32 search.completed 800.7ms Concept | 10 results | top: 0.628 | embed: 794.0ms | search: 4.3ms | hydrate: 2.4ms
20:00:31 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
16:11:02 search.completed 840.0ms Concept | 10 results | top: 0.617 | embed: 829.9ms | search: 6.8ms | hydrate: 3.3ms
16:11:01 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 28 chars
15:29:12 search.completed 795.5ms Concept | 10 results | top: 0.685 | embed: 788.1ms | search: 5.0ms | hydrate: 2.4ms
15:29:11 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 14 chars
14:33:41 search.completed 766.1ms Concept | 10 results | top: 0.628 | embed: 756.7ms | search: 5.9ms | hydrate: 3.5ms
14:33:40 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 23 chars
12:52:51 search.completed 805.4ms Concept | 10 results | top: 0.558 | embed: 797.3ms | search: 5.2ms | hydrate: 2.8ms
12:52:50 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
11:21:02 search.completed 809.7ms Concept | 10 results | top: 0.691 | embed: 801.3ms | search: 5.4ms | hydrate: 3.0ms
11:21:01 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 32 chars
07:13:15 search.completed 742.0ms Concept | 10 results | top: 0.628 | embed: 734.2ms | search: 5.3ms | hydrate: 2.5ms
07:13:14 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
05:37:11 search.completed 799.7ms Concept | 10 results | top: 0.685 | embed: 789.9ms | search: 6.6ms | hydrate: 3.1ms
05:37:11 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 14 chars
00:07:51 search.completed 770.1ms Concept | 10 results | top: 0.558 | embed: 763.0ms | search: 4.8ms | hydrate: 2.3ms
00:07:50 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
23:15:22 search.completed 785.9ms Concept | 10 results | top: 0.628 | embed: 778.3ms | search: 4.8ms | hydrate: 2.8ms
23:15:21 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
21:46:11 search.completed 778.3ms Concept | 10 results | top: 0.628 | embed: 770.5ms | search: 5.2ms | hydrate: 2.6ms
21:46:10 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 23 chars
21:13:34 search.completed 878.9ms Concept | 10 results | top: 0.617 | embed: 871.5ms | search: 5.2ms | hydrate: 2.3ms
21:13:34 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 28 chars
13:30:53 search.completed 100.2ms Concept | 10 results | top: 0.685 | embed: 93.2ms | search: 4.7ms | hydrate: 2.3ms
13:30:53 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 14 chars
13:30:34 search.completed 103.4ms Concept | 10 results | top: 0.628 | embed: 95.7ms | search: 5.2ms | hydrate: 2.6ms
13:30:34 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
13:30:32 search.completed 815.3ms Concept | 10 results | top: 0.617 | embed: 808.2ms | search: 4.8ms | hydrate: 2.3ms
13:30:31 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 28 chars
02:22:30 search.completed 824.5ms Concept | 10 results | top: 0.685 | embed: 815.4ms | search: 6.4ms | hydrate: 2.7ms
02:22:29 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 14 chars
02:13:49 search.completed 775.0ms Concept | 10 results | top: 0.691 | embed: 767.4ms | search: 5.3ms | hydrate: 2.3ms
02:13:48 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 32 chars
01:08:55 search.completed 756.6ms Concept | 10 results | top: 0.691 | embed: 749.5ms | search: 4.6ms | hydrate: 2.4ms
01:08:54 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 32 chars
00:56:25 search.completed 769.2ms Concept | 10 results | top: 0.685 | embed: 762.5ms | search: 4.2ms | hydrate: 2.5ms
00:56:24 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 14 chars
00:47:01 search.completed 802.8ms Concept | 10 results | top: 0.628 | embed: 793.5ms | search: 6.3ms | hydrate: 3.0ms
00:47:00 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars

Telemetry API

GET /api/telemetry

Aggregated statistics for the specified period

GET /api/telemetry/recent

Recent event stream with optional type filter

Artisan Command
php artisan brim:telemetry stats --period=24h

Telemetry Configuration

telemetry.enabled true
telemetry.store.enabled true
telemetry.logging.enabled false
telemetry.debug false
telemetry.sample_rate 1
telemetry.store.retention_days 30