Observability

Brim Internals

Deep visibility into the semantic search pipeline execution

Pipeline Architecture

BRIM SEMANTIC SEARCH PIPELINE

EMBEDDING GENERATION

1. Text Extraction Model::toEmbeddableText()
2. Text Chunking ~0 chunks
3. Embedding (Ollama) 0ms avg
4. Storage (pgvector) 0ms avg
Total 0ms avg

SEMANTIC SEARCH

1. Query Embedding 222.59ms avg
2. Vector Search (cosine) 4.63ms avg
3. Model Hydration 2.47ms avg
4. Results avg 10 @ 0.635
Total 229.68ms avg

Embedding Generation

0
Total Generated
0ms
Avg Duration
Embedding Time (Ollama) 0ms
Storage Time (pgvector) 0ms
Total Chunks Created 0
Min / Max Duration 0ms / 0ms

Semantic Search

6
Total Searches
229.68ms
Avg Duration
Query Embedding Time 222.59ms
Vector Search Time 4.63ms
Model Hydration Time 2.47ms
Avg Results / Top Score 10 / 0.635
Min / Max Duration 98.8ms / 814.36ms

Pipeline Timing Breakdown

Embedding Generation

Text Chunking ~5ms
chunk
Ollama Embedding 0ms
embed
pgvector Insert 0ms
store

Semantic Search

Query Embedding 222.59ms
embed
Vector Search (cosine) 4.63ms
search
Model Hydration 2.47ms
hydrate

Event Stream

Time Event Duration Details
04:20:44 search.completed 146.0ms Concept | 10 results | top: 0.628 | embed: 137.9ms | search: 5.2ms | hydrate: 2.9ms
04:20:44 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 23 chars
04:20:17 search.completed 98.8ms Concept | 10 results | top: 0.628 | embed: 92.0ms | search: 4.3ms | hydrate: 2.5ms
04:20:17 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
04:20:10 search.completed 110.1ms Concept | 10 results | top: 0.617 | embed: 102.7ms | search: 4.8ms | hydrate: 2.5ms
04:20:10 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 28 chars
04:19:59 search.completed 101.8ms Concept | 10 results | top: 0.558 | embed: 95.1ms | search: 4.6ms | hydrate: 2.1ms
04:19:58 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
04:19:38 search.completed 107.1ms Concept | 10 results | top: 0.685 | embed: 99.6ms | search: 4.8ms | hydrate: 2.7ms
04:19:38 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 14 chars
04:15:39 search.completed 814.4ms Concept | 10 results | top: 0.691 | embed: 808.1ms | search: 4.1ms | hydrate: 2.2ms
04:15:38 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 32 chars
23:45:48 search.completed 753.2ms Concept | 10 results | top: 0.628 | embed: 747.2ms | search: 3.8ms | hydrate: 2.2ms
23:45:48 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
23:40:10 search.completed 764.4ms Concept | 10 results | top: 0.558 | embed: 757.4ms | search: 4.7ms | hydrate: 2.2ms
23:40:10 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
23:29:07 search.completed 794.9ms Concept | 10 results | top: 0.617 | embed: 786.8ms | search: 5.6ms | hydrate: 2.6ms
23:29:06 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 28 chars
12:53:46 search.completed 774.5ms Concept | 10 results | top: 0.628 | embed: 768.7ms | search: 3.8ms | hydrate: 2.0ms
12:53:45 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
18:16:58 search.completed 122.3ms Concept | 10 results | top: 0.628 | embed: 104.6ms | search: 5.6ms | hydrate: 12.1ms
18:16:58 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
18:16:38 search.completed 818.8ms Concept | 10 results | top: 0.628 | embed: 810.3ms | search: 5.4ms | hydrate: 3.1ms
18:16:38 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
02:22:53 search.completed 773.6ms Concept | 10 results | top: 0.628 | embed: 765.5ms | search: 5.7ms | hydrate: 2.4ms
02:22:52 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 23 chars
01:31:43 search.completed 721.8ms Concept | 10 results | top: 0.628 | embed: 716.3ms | search: 3.8ms | hydrate: 1.8ms
01:31:42 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 23 chars
00:14:15 search.completed 779.8ms Concept | 10 results | top: 0.558 | embed: 771.1ms | search: 6.2ms | hydrate: 2.5ms
00:14:14 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
15:45:26 search.completed 97.7ms Concept | 10 results | top: 0.685 | embed: 90.6ms | search: 4.8ms | hydrate: 2.4ms
15:45:26 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 14 chars
15:41:12 search.completed 83.4ms Concept | 10 results | top: 0.628 | embed: 78.4ms | search: 3.5ms | hydrate: 1.6ms
15:41:12 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 23 chars
15:41:10 search.completed 92.8ms Concept | 10 results | top: 0.628 | embed: 86.2ms | search: 4.6ms | hydrate: 2.1ms
15:41:10 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 23 chars
15:41:08 search.completed 100.2ms Concept | 10 results | top: 0.628 | embed: 93.2ms | search: 4.8ms | hydrate: 2.2ms
15:41:08 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 23 chars
15:39:02 search.completed 134.4ms Concept | 10 results | top: 0.492 | embed: 124.5ms | search: 6.8ms | hydrate: 3.1ms
15:39:02 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 4 chars
15:38:57 search.completed 67.2ms Concept | 10 results | top: 0.531 | embed: 61.3ms | search: 4.0ms | hydrate: 1.9ms
15:38:57 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 5 chars
15:38:52 search.completed 69.8ms Concept | 10 results | top: 0.485 | embed: 63.5ms | search: 4.2ms | hydrate: 2.1ms
15:38:52 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 2 chars
15:38:44 search.completed 88.5ms Concept | 10 results | top: 0.442 | embed: 81.1ms | search: 4.9ms | hydrate: 2.4ms
15:38:44 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 4 chars
15:36:31 search.completed 498.2ms Concept | 10 results | top: 0.617 | embed: 492.2ms | search: 4.0ms | hydrate: 2.1ms
15:36:30 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 28 chars
06:44:52 search.completed 720.0ms Concept | 10 results | top: 0.691 | embed: 712.9ms | search: 5.0ms | hydrate: 2.2ms
06:44:51 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 32 chars

Telemetry API

GET /api/telemetry

Aggregated statistics for the specified period

GET /api/telemetry/recent

Recent event stream with optional type filter

Artisan Command
php artisan brim:telemetry stats --period=24h

Telemetry Configuration

telemetry.enabled true
telemetry.store.enabled true
telemetry.logging.enabled false
telemetry.debug false
telemetry.sample_rate 1
telemetry.store.retention_days 30