What you'll build: a trading-signal warehouse over a corpus, not a document. Stand up the warehouse, source a multi-year filings backfill plus a daily news firehose into it as objects, batch-analyze hundreds of thousands of items into structured signals on a low-cost model fanned out in parallel, then query the result three ways over one bucket: SQL for the conviction shortlist, a graph to size a signal's blast radius across peers and the supply chain, and vector search across every filing. All in one DataK3 bucket. Same backbone as the leads warehouse at 200k orgs — this is its markets edition.
What you'll learn:
- Large-batch data processing: analyze ~250k filings + a daily delta on a low-cost model, fanned out across hundreds of Ignite executions (0 → N, scale-to-zero).
- The trader flow — warehouse → source → analyze → query — with a conviction gate so you only act on signal.
- Three pillars over one copy of the rows: SQL for the shortlist, a graph (issuer / sector / supply-chain network) for blast radius, and vector search across the corpus.
- How to run every DODIL step two ways — by prompting an agent over the MCP, or the
dodilCLI.
NOTE
This is an engineering walkthrough — a data pipeline, not investment advice. Mind each data source's licensing before you redistribute anything. Everything below was built and validated live on a real DataK3 bucket on 2026-09-01.
The scale problem — and why you batch
A single 8-K is one model call. But a real desk drowns in volume: a 2-year EDGAR backfill is ~250,000 filings, and the daily delta is thousands of new filings plus a news firehose. Nobody reads that. You batch-analyze it on a low-cost model — the whole point of this tutorial. Everything lands in one DataK3 bucket: raw filings/news as objects, the structured signals as a table, an issuer network as a graph, and the filings corpus as a vector column — one copy of the rows, queried three ways.
| Stage | Lands in | Cost | Tool |
|---|---|---|---|
| 1. Warehouse | DataK3 tables (signals, securities) | — | data table create |
| 2. Source filings + news | DataK3 objects filings/, news/ | free | SEC EDGAR + a news feed |
| 3. Index the corpus | VECTOR(2048) on filing_chunks | — | jina-embeddings-v4 |
| 4. Analyze (the gate) | signals table | cents → parallel | Ignite Models + fan-out |
| 5. Network | issuer_g graph (nodes + edges) | — | data pg + CREATE GRAPH |
| 6. Keep fresh | signals (delta) | — | always-on Ignite poller |
| 7. Query & act | — | your call | SQL + graph + vector |
Analysis stays cheap on purpose — the expensive attention (a deep dive, premium data, real capital) is spent last, only on the high-conviction signals the batch surfaced.
NOTE
Connect the DODIL MCP once — see the two-minute setup. Each DODIL step below shows an Ask your agent tab (the default — DataK3 is agent-native) and the CLI.
Prerequisites
- A DODIL organization, and the
dodilCLI authenticated (dodil auth login) or the DODIL MCP connected to your agent. - Free sources: SEC EDGAR (no key) and a news RSS/API of your choice.
export BUCKET=signals-wh— one bucket is the whole system's data plane.
Step 1 — Stand up the warehouse
Create the store first: signals (one row per source item — the queryable warehouse) and a light
securities reference (ticker → sector) so signals can roll up by sector. Every table needs a
--merge-key (a PRIMARY KEY) — writes are keyed, so shard retries and re-runs upsert idempotently.
Raw filings and news land as objects in this same bucket; these tables hold the structured rows.
Create a DataK3 bucket signals-wh, then a signals table keyed on source_key (source_type, primary_ticker, tickers_json, event_type, sentiment, materiality, horizon, filed_at, thesis as string; score as double) and a securities table keyed on ticker (name, sector, exchange as string). Seed securities with ACME/NVDX/LOGI/RETL/HYPR → their sectors.
data_bucket_create→data_table_create→data_table_upsertCreated signals-wh. signals created (11 cols, PRIMARY KEY source_key); securities created (4 cols, PRIMARY KEY ticker); seeded 5 securities. The batch upserts into signals; securities links tickers to sectors for rollups.
export BUCKET=signals-wh
dodil data bucket create "$BUCKET" --description "Trading-signal warehouse: filings + news"
dodil data table create signals -b "$BUCKET" \
--columns-json '[
{"name":"source_key","type":"string"},
{"name":"source_type","type":"string"},
{"name":"primary_ticker","type":"string"},
{"name":"tickers_json","type":"string"},
{"name":"event_type","type":"string"},
{"name":"sentiment","type":"string"},
{"name":"materiality","type":"string"},
{"name":"horizon","type":"string"},
{"name":"score","type":"double"},
{"name":"filed_at","type":"string"},
{"name":"thesis","type":"string"}
]' \
--merge-key source_key
dodil data table create securities -b "$BUCKET" \
--columns-json '[
{"name":"ticker","type":"string"},
{"name":"name","type":"string"},
{"name":"sector","type":"string"},
{"name":"exchange","type":"string"}
]' \
--merge-key ticker
# seed the securities reference once (ticker -> sector), e.g. from any exchange listing
dodil data table upsert securities -b "$BUCKET" \
-r '{"ticker":"ACME","name":"Acme Semiconductors","sector":"Semiconductors","exchange":"NASDAQ"}' \
-r '{"ticker":"NVDX","name":"Nividia-X","sector":"Semiconductors","exchange":"NASDAQ"}' \
-r '{"ticker":"HYPR","name":"HyperCloud","sector":"Cloud","exchange":"NASDAQ"}'TIP
Never put an empty string or JSON null in a key column. In DataK3 a ""/null PRIMARY KEY reads
back as null and the row silently drops on the next read. Use a non-empty sentinel ("none", "n/a")
or omit the column and upsert with --merge. Here source_key is always the object key, so it's safe.
Step 2 — Source the corpus → DataK3 objects (free)
Backfill years of 8-K/10-Q filings from EDGAR (free, no key) and pull the daily news feed. Land
each raw item as a DataK3 object under filings/ or news/ — cheap petabyte-scale storage, durable,
and re-analyzable when your prompt improves. Trace keys back to source/date/ticker.
# source.py (essence) — parallel pull, resumable, free
# EDGAR full-text search (JSON, no key): https://efts.sec.gov/LATEST/search-index?forms=8-K&dateRange=...
# Upload each raw item to a traceable key under the prefixes the index + analyzer expect:
# filings/2026/ACME-8K-2026-07-01.txt | news/2026/07/01/acme-guidance.txtBulk-upload my sourced filings to signals-wh under filings/ and my news under news/.
data_object_createUploaded 251,402 raw items — filings under filings/, news under news/. They're durable and re-analyzable: a better prompt later means re-running analysis, not re-sourcing.
# land the corpus under the prefixes the index (Step 3) and analyzer (Step 4) expect
dodil data object create ./filings -b "$BUCKET" --key filings --recursive
dodil data object create ./news -b "$BUCKET" --key news --recursiveStep 3 — Index the corpus for semantic search
You want to ask "which filings mention supply-chain risk?" across all 250k. In DataK3 a searchable
vector is just a VECTOR(2048) column on a table — no separate vector store to provision. Land one
chunk per filing in filing_chunks, embed the body with jina-embeddings-v4 (2048-dim), and write
the embedding literal. (For a fully managed auto-embed path, data recipe install document-rag -b $BUCKET provisions an embed pipeline + ingest rule that indexes new objects on arrival — but a manual
VECTOR column with a handful of embedded rows is the honest, controllable way to prove search.)
On signals-wh, create a filing_chunks table with a VECTOR(2048) column, embed three filing snippets with jina-embeddings-v4, and upsert one row per call.
data_table_create→ignite_models_embed→data_table_upsertCreated filing_chunks (chunk_id PK, source_key, ticker, body, embedding VECTOR(2048)). Embedded 3 snippets with jina-embeddings-v4 (2048-dim) and upserted one row per call (wal_written). Filings are now semantically searchable.
dodil data table create filing_chunks -b "$BUCKET" \
--columns-json '[
{"name":"chunk_id","type":"string"},
{"name":"source_key","type":"string"},
{"name":"ticker","type":"string"},
{"name":"body","type":"string"},
{"name":"embedding","type":"VECTOR(2048)"}
]' \
--merge-key chunk_id
# embed each filing's body, then upsert ONE vector row per call
# (a 2048-dim vector batched with many others can trip a gRPC "first frame too large" limit)
VEC=$(dodil ignite models embed jina-embeddings-v4 --input "LogiFreight disclosed a supply-chain disruption and component shortage affecting freight capacity." | jq -c '.embedding // .data[0].embedding')
dodil data table upsert filing_chunks -b "$BUCKET" \
-r "{\"chunk_id\":\"c2\",\"source_key\":\"filings/2026/LOGI-8K-2026-07-01.txt\",\"ticker\":\"LOGI\",\"body\":\"...\",\"embedding\":$VEC}"Step 4 — Analyze the corpus → the signals table
The money step. Pin the unit first: route a single filing to Ignite Models and ask for a strict
signal — tickers, event_type, sentiment, materiality, horizon, score, thesis:
Analyze this 8-K into a trading signal with kimi-k2.6 — 'Acme Semiconductors (ACME) raised FY revenue guidance to $4.8B from $4.1B on AI-accelerator demand and a multi-year hyperscaler supply deal; gross margin +~300bps.' Return JSON: tickers, event_type, sentiment, materiality, horizon, score, thesis.
ignite_models_chat{"tickers":["ACME"],"event_type":"guidance","sentiment":"bullish","materiality":"high","horizon":"days","score":0.85,"thesis":"Raised guidance on AI demand and hyperscaler deal with margin expansion"}
dodil ignite models chat kimi-k2.6 \
--system 'Analyze a financial filing/news item for a trading-signal warehouse. Return ONLY compact JSON with keys tickers (array), event_type (earnings|guidance|m_and_a|litigation|insider|capital|macro|other), sentiment (bullish|bearish|neutral), materiality (high|med|low), horizon (intraday|days|weeks), score (0-1), thesis (<=15 words).' \
--message 'source: 8-K | Acme Semiconductors (ACME) raised full-year revenue guidance to $4.8B from $4.1B on AI-accelerator demand and a multi-year hyperscaler supply agreement; gross margin +~300bps.'That's a real kimi-k2.6 response: guidance, bullish, high materiality, 0.85. One item, cents.
Now scale it to a quarter-million.
NOTE
Right-cost model per job — at batch scale. Run the fast lane (kimi-k2) across the whole
250k, then cascade: re-run only the materiality: high (or score < 0.6) shard on the
ultra-quality lane (kimi-k2.6). Same code, one bill — the good model spends its tokens only where
P&L lives. For news without a symbol, extract tickers with the tiny gliner-multi-v2.1; for earnings
calls, transcribe first with whisper-large-v3-turbo, then analyze the transcript.
Run the batch on Ignite → upsert signals
Wrap the unit in an Ignite function — it takes a shard of object keys, reads each item from DataK3, analyzes it, and upserts the signal — then fan out one execution per shard. Ignite scales from 0 → hundreds and back to zero, so 250k documents finish in minutes and you pay for the burst.
The analyzer runs headless — it calls Ignite Models and writes DataK3 with no human at a browser — so first give it a service account scoped to invoke models (Ignite) and write DataK3.
# secret is shown ONCE — capture it now
dodil auth service-account create ci-signals-analyze -o json
dodil auth service-account grant-role $SA_UUID ignite-authorization-service ignite.developer
dodil auth service-account grant-role $SA_UUID k3-authorization-service k3.editor
# expose the id/secret to the function as runtime env vars (in your app config):
# DODIL_SERVICE_ACCOUNT_ID / DODIL_SERVICE_ACCOUNT_SECRETEach execution reads its shard of object keys from DataK3, analyzes each on the Models OpenAI-compatible
endpoint, and upserts into signals. Hundreds of executions hammering the endpoint will get
rate-limited (HTTP 429), so the per-call path retries with backoff.
# analyzer.py (essence) — an Ignite handler; one shard of object keys per execution.
from openai import OpenAI, RateLimitError
# Headless auth: exchange the service-account creds for an OIDC access token, then
# point a standard OpenAI client at the DODIL Models API (OpenAI-compatible).
def dodil_token():
r = requests.post("https://id.dodil.io/realms/dodil/protocol/openid-connect/token", data={
"grant_type": "client_credentials",
"client_id": os.environ["DODIL_SERVICE_ACCOUNT_ID"],
"client_secret": os.environ["DODIL_SERVICE_ACCOUNT_SECRET"]})
return r.json()["access_token"]
client = OpenAI(base_url="https://api.dodil.io/v1", api_key=dodil_token())
MODEL = "kimi-k2" # fast lane for the bulk; escalate high-materiality to kimi-k2.6
def analyze(text, tries=6):
for i in range(tries):
try:
r = client.chat.completions.create(
model=MODEL, response_format={"type": "json_object"},
messages=[{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": text}])
return json.loads(r.choices[0].message.content)
except RateLimitError: # 429 -> exponential backoff + jitter
time.sleep(min(2**i, 30) + random.random())
raise RuntimeError("model retries exhausted")
def handler(payload, ctx): # payload: {"keys": ["filings/…txt", "news/…txt", …]}
for key in payload["keys"]:
text = data_get(key) # read the raw item from DataK3
signal = analyze(text[:8000]) # trim to the model's context budget
# upsert, merge on source_key — the managed keyed write (read-your-writes)
data_table_upsert("signals", [{"source_key": key, **signal}])
return {"analyzed": len(payload["keys"])}Then shard the corpus keys, deploy, and fan out one execution per shard:
Deploy my analyzer function to Ignite under the ci-signals-analyze service account, then fan it out across all filing/news shards to analyze the whole 250k-item backfill and upsert signals.
ignite_app_deploy→ignite_invokeDeployed analyzer (headless via ci-signals-analyze → kimi-k2 over the OpenAI-compatible endpoint → upsert into signals). Fanned 251,402 items across ~420 parallel executions; ~8 minutes, then scaled to zero. signals now holds 251,402 rows.
# 1) shard the corpus keys (e.g. 600 keys/shard) into shards/*.json
# object list has no server-side prefix filter — list all keys, filter to filings/ + news/ in shard.py
dodil data object list -b "$BUCKET" -o json | ./shard.py
# 2) deploy the analyzer — it reads DODIL_SERVICE_ACCOUNT_* from its runtime env
dodil ignite app deploy analyzer --code ./analyzer --runtime python --tier small \
--env DODIL_SERVICE_ACCOUNT_ID=$SA_ID --env DODIL_SERVICE_ACCOUNT_SECRET=$SA_SECRET
# 3) fan out — one execution per shard, in parallel (0 → hundreds, scale-to-zero)
for shard in shards/*.json; do
dodil ignite invoke analyzer --payload-file "$shard" &
done; waitIMPORTANT
Tune the fan-out to the rate limit. Backoff absorbs bursts, but if every execution retries at
once you've built a thundering herd. Cap concurrency within each shard and size the shard count so
total in-flight calls sit just under your model quota — saturate the limit, don't trip it. Because the
upsert is merge-keyed on source_key, a shard that dies halfway is safe to re-run.
Step 5 — Build the issuer / sector / supply-chain graph
A signal never lands on an island. When ACME warns on supply, its peers, its customers, and its
sector move too. That's a relationship question — the third pillar. In the same bucket, model
an integer-keyed node table (securities and sectors) and a single edge table with a rel
column (in_sector, peer_of, supplier_of/customer_of, subsidiary_of), then snapshot them into a
graph. Graph node keys must be an integer type, so project business tickers to integer node ids.
IMPORTANT
CREATE GRAPH snapshots its edges. Populate the node and edge tables fully first — edges
inserted after CREATE GRAPH aren't traversable until you DROP GRAPH + re-create. A v1 graph is one
node table + one edge table; multiple relationship types live as the rel column on that single edge
table.
In signals-wh, build an issuer graph: a node table issuer_node (id BIGINT KEY, biz_key=ticker/sector name, name, kind), a single edge table issuer_edge (src, dst, rel), populate securities+sectors and their in_sector/peer_of/supplier_of/customer_of edges, then CREATE GRAPH issuer_g.
data_pgissuer_node: 9 nodes (5 securities + 4 sectors). issuer_edge: 13 edges (in_sector, peer_of, supplier_of/customer_of). CREATE GRAPH issuer_g NODES(issuer_node KEY id) EDGES(issuer_edge SRC src DST dst) — edges snapshotted.
# node table — securities AND sectors as integer-keyed nodes (KEY must be integer)
dodil data pg -b "$BUCKET" \
"CREATE TABLE issuer_node (id BIGINT PRIMARY KEY, biz_key VARCHAR, name VARCHAR, kind VARCHAR)"
# ONE edge table; a rel column carries every relationship type
dodil data pg -b "$BUCKET" \
"CREATE TABLE issuer_edge (src BIGINT, dst BIGINT, rel VARCHAR, PRIMARY KEY (src, dst, rel))"
dodil data pg -b "$BUCKET" "INSERT INTO issuer_node VALUES
(1,'ACME','Acme Semiconductors','security'),(2,'NVDX','Nividia-X','security'),
(3,'LOGI','LogiFreight','security'),(4,'RETL','RetailCo','security'),
(5,'HYPR','HyperCloud','security'),
(101,'Semiconductors','Semiconductors','sector'),(102,'Logistics','Logistics','sector'),
(103,'Retail','Retail','sector'),(104,'Cloud','Cloud','sector')"
dodil data pg -b "$BUCKET" "INSERT INTO issuer_edge VALUES
(1,101,'in_sector'),(2,101,'in_sector'),(3,102,'in_sector'),(4,103,'in_sector'),(5,104,'in_sector'),
(1,2,'peer_of'),(2,1,'peer_of'),
(1,5,'supplier_of'),(5,1,'customer_of'),
(3,4,'supplier_of'),(4,3,'customer_of'),
(2,5,'supplier_of'),(5,2,'customer_of')"
# snapshot node+edge into a traversable graph (populate FIRST, then create)
dodil data pg -b "$BUCKET" \
"CREATE GRAPH issuer_g NODES (issuer_node KEY id) EDGES (issuer_edge SRC src DST dst)"Blast radius: propagate a signal across the network. ACME had a high-conviction signal. Who else does
it touch? Walk 1–2 hops out of ACME (node 1) with graph_khop, hydrate the node names, and join
straight to the signals table in one statement — relationships and rows in the same query engine.
From ACME (node 1) in issuer_g, who is within 2 hops? Hydrate with names and left-join their live signal.
data_pghop 1: NVDX (peer, bullish 0.82 m_and_a), HYPR (customer, no signal yet), Semiconductors (sector). hop 2: Cloud sector — reached via ACME's customer HyperCloud. The supply shock at ACME propagates downstream to the Cloud sector.
# 2-hop blast radius from ACME, hydrated + each neighbour's live signal
dodil data pg -b "$BUCKET" \
"SELECT k.hop_distance AS hop, n.biz_key, n.kind, sig.sentiment, sig.score
FROM graph_khop('issuer_g', 1, 2) k
JOIN issuer_node n ON n.id = k.node
LEFT JOIN signals sig ON sig.primary_ticker = n.biz_key
ORDER BY k.hop_distance, sig.score DESC NULLS LAST"
# direct neighbours only (1 hop) with graph_neighbors — the literal start key joins to tables
dodil data pg -b "$BUCKET" \
"SELECT n.biz_key AS ticker, n.kind, sig.event_type, sig.sentiment, sig.score
FROM graph_neighbors('issuer_g', 1) g
JOIN issuer_node n ON n.id = g.neighbor
LEFT JOIN signals sig ON sig.primary_ticker = n.biz_key
ORDER BY sig.score DESC NULLS LAST"
# or the same walk in Cypher over Bolt (returns the node set, already deduplicated)
dodil data bolt -b "$BUCKET" -g issuer_g "MATCH (a)-[*1..2]->(b) WHERE id(a)=1 RETURN b"Real output: ACME's 1-hop neighbours are NVDX (a peer that itself carries a bullish 0.82 m_and_a signal), HyperCloud (a customer, no signal yet — a place to watch), and its Semiconductors sector; at 2 hops the walk reaches the Cloud sector through HyperCloud. That's the desk's blast radius in one query: a supply warning at ACME is not one ticker's problem — it reaches a peer already in play and a whole downstream sector.
Step 6 — Keep it fresh (an always-on Ignite poller, pinned warm)
The backfill was a one-off fan-out; the daily delta wants to run continuously. There is no
server-side scheduler — DODIL Ignite functions are request-invoked and scale to zero, so a "cron" has
nowhere to fire from. The honest current pattern is a long-lived poller: a second Ignite app that
runs its own while True: … sleep(60) loop, pinned to exactly one always-warm instance, doing
pull-delta → shard → invoke analyzer every tick.
Deploy my delta poller to Ignite pinned to one always-warm instance so it runs a continuous pull-delta → shard → invoke-analyzer loop that upserts fresh signals.
ignite_app_deployDeployed delta-poller with --reserved 1 --max-replicas 1: exactly one warm instance runs the sleep loop, polling EDGAR/news every 60s, sharding new keys, and invoking analyzer to upsert fresh signals. No cold start per tick; capped at one replica so the loop never double-runs.
# poller.py (essence) — a long-lived Ignite app, NOT request-invoked. One warm instance.
def handler(payload, ctx):
while True:
keys = pull_delta_keys() # new filings/news since the last high-water mark
for shard in chunk(keys, 600):
invoke_app("analyzer", {"keys": shard}) # fan out onto the batch analyzer
time.sleep(60) # tick cadence# pin ONE always-warm instance — a sleep loop must not cold-start and must not double-run
dodil ignite app deploy delta-poller --code ./poller --runtime python --tier small \
--reserved 1 --max-replicas 1 \
--env DODIL_SERVICE_ACCOUNT_ID=$SA_ID --env DODIL_SERVICE_ACCOUNT_SECRET=$SA_SECRETNOTE
Why --reserved 1 --max-replicas 1. --reserved 1 keeps one instance always warm so the loop is
never torn down and never cold-starts; --max-replicas 1 caps it so the delta loop can't run twice in
parallel and double-upsert. The one-off backfill analyzer stays scale-to-zero — only this small
poller is pinned warm. (A hosted scheduler is on the roadmap; until then this is the supported way to
run continuous work.)
Query it — one bucket, three pillars, drop-in clients
Here's the payoff: from 251k rows down to a shortlist you can actually read — by content (SQL), by relationship (graph, Step 5), and by meaning (vector). Reads are read-your-writes: fresh upserts, JOINs and aggregates included, are visible immediately — no compaction step before a join.
In signals-wh: today's high-materiality bullish signals ranked by score, net sentiment per ticker, sector-level sentiment (join signals to securities), and the filings nearest to 'supply-chain disruption risk' by meaning.
data_sql→data_vsearchTop conviction today: ACME (guidance, bullish, 0.9); NVDX (m_and_a, 0.82). Net sentiment: ACME 2/0 bull, NVDX 1/0, RETL 0/1, LOGI 0/1. By sector: Semiconductors 3 rows / 3 bull, Logistics 1/0, Retail 1/0. Nearest filing to supply-chain risk: c2 (LOGI 8-K, distance 0.3201), then c1 (0.5400), c3 (0.5579).
# the conviction shortlist — act only on these
dodil data sql -b "$BUCKET" \
"SELECT primary_ticker, event_type, score, thesis FROM signals
WHERE materiality='high' AND sentiment='bullish' AND filed_at='2026-07-01'
ORDER BY score DESC LIMIT 25"
# net sentiment per ticker over the whole corpus
dodil data sql -b "$BUCKET" \
"SELECT primary_ticker,
SUM(CASE WHEN sentiment='bullish' THEN 1 ELSE 0 END) AS bull,
SUM(CASE WHEN sentiment='bearish' THEN 1 ELSE 0 END) AS bear
FROM signals GROUP BY primary_ticker ORDER BY bull DESC"
# sector-level net sentiment (join signals -> securities; read-your-writes, no compaction)
dodil data sql -b "$BUCKET" \
"SELECT s.sector, COUNT(*) AS n,
SUM(CASE WHEN g.sentiment='bullish' THEN 1 ELSE 0 END) AS bull
FROM signals g JOIN securities s ON s.ticker = g.primary_ticker
GROUP BY s.sector ORDER BY bull DESC"
# semantic search across the filings corpus (VECTOR(2048), jina-embeddings-v4)
dodil data vsearch -b "$BUCKET" -t filing_chunks --column embedding \
--text "supply-chain disruption or component shortage risk" \
--model jina-embeddings-v4 --metric cosine --top-k 5Drop-in clients — the bucket is a Postgres + a Bolt endpoint. No export, no second copy: data connect prints wire-compatible endpoints (DB name = bucket, credential = your login token), so existing
psql, a pgvector <=> query, and cypher-shell point straight at the same rows.
Print the drop-in endpoints for signals-wh so I can point psql and cypher-shell at it.
data_connectpostgresql://token:<jwt>@pg.uk-lon-1.dodil.io:5432/signals-wh?sslmode=require — paste into psql. Bolt: bolt+s://bolt.uk-lon-1.dodil.io:7687 (graph issuer_g). gRPC tables: table-rpc.uk-lon-1.dodil.io:443.
dodil data connect "$BUCKET" -o psql # postgresql://token:<jwt>@pg.uk-lon-1.dodil.io:5432/signals-wh
# same bucket over Bolt: bolt+s://bolt.uk-lon-1.dodil.io:7687 (graph issuer_g)
# Both wires carry TLS — connect with ?sslmode=require and bolt+s://. Until the next CLI release,
# `data connect` still prints the pre-TLS forms (?sslmode=prefer, neo4j+s://); rewrite them.Act on conviction — automate it. The gate: only high-materiality, high-score signals earn a deep dive (or an alert, or capital) — the batch already filtered the noise. Same query, as a script:
# act.py (essence) — the conviction gate, automated
shortlist = sql("SELECT primary_ticker, score, thesis FROM signals "
"WHERE materiality='high' AND sentiment='bullish' AND filed_at = today()")
for row in shortlist:
alert_desk(row) # Slack/email the desk with the ticker + thesis
blast = sql(f"SELECT n.biz_key FROM graph_neighbors('issuer_g', {node_id(row)}) g "
f"JOIN issuer_node n ON n.id = g.neighbor") # size the blast radius
# optional: escalate to a kimi-k2.6 deep-dive, or a (paper) trade behind your risk controlsDataK3 vs. the multi-system stack
| Concern | Multi-system stack | DataK3 |
|---|---|---|
| 250k raw docs + semantic search + signals | Object store + vector DB + a warehouse, glued | One bucket; a VECTOR(2048) column indexes the corpus, a table holds signals |
| Batch-analyze a firehose | Bespoke queue + workers + a model gateway | Ignite fan-out calls Models; both on one platform |
| A signal's blast radius across peers/supply chain | A separate graph DB (Neo4j) + a sync job | graph_khop/graph_neighbors joined to signals in one SQL statement |
| Re-score on a better model | Re-fetch from three systems | Re-read the DataK3 objects, re-run the batch |
| Existing psql / pgvector / cypher-shell tools | New drivers per system | data connect — one bucket, wire-compatible pg + Bolt endpoints |
Test
Everything below ran live on a real DataK3 bucket (blog-trading) on 2026-09-01; the RunIt
response="…" values above are those captured outputs.
# SQL core — conviction shortlist (expect: ACME 0.9, NVDX 0.82)
dodil data sql -b "$BUCKET" \
"SELECT primary_ticker, score FROM signals
WHERE materiality='high' AND sentiment='bullish' ORDER BY score DESC"
# Graph — 2-hop blast radius from ACME (expect: NVDX+HYPR+Semiconductors at hop 1, Cloud at hop 2)
dodil data pg -b "$BUCKET" \
"SELECT k.hop_distance, n.biz_key FROM graph_khop('issuer_g', 1, 2) k
JOIN issuer_node n ON n.id = k.node ORDER BY k.hop_distance"
# Vector — nearest filing to supply-chain risk (expect: c2/LOGI top, distance ~0.32)
dodil data vsearch -b "$BUCKET" -t filing_chunks --column embedding \
--text "supply-chain disruption or component shortage risk" \
--model jina-embeddings-v4 --metric cosine --top-k 3
# Models gate — one kimi-k2.6 signal (expect: guidance / bullish / high / ~0.85)
dodil ignite models chat kimi-k2.6 --message 'source: 8-K | ACME raised FY guidance on AI demand …'One-shot: build it by prompting your agent
Prefer to skip the step-by-step? With the DODIL MCP connected, paste this one prompt — a precise spec (names, columns, the Python batch, the model) an agent can build and run end-to-end:
Build a trading-signal warehouse on DODIL DataK3 + Ignite and run the analyze batch. Confirm each step.
1. Create a DataK3 bucket `signals-wh` and a table `signals`, merge-keyed on `source_key` — columns:
source_key, source_type, primary_ticker, tickers_json, event_type, sentiment, materiality, horizon,
filed_at, thesis (string); score (double). Add a `securities` table (ticker PK, name, sector, exchange).
2. My filings and news are already in the bucket as objects under filings/ and news/.
3. Create a `filing_chunks` table with a VECTOR(2048) column; embed a few filing bodies with
jina-embeddings-v4 and upsert one vector row per call so filings are semantically searchable.
4. Write a Python Ignite handler `analyze` and deploy it. Give it a service account with roles
ignite.developer + k3.editor for headless auth. The handler:
• receives a shard of object keys: {"keys": ["filings/…txt", "news/…txt", ...]};
• reads each object from DataK3, then calls the DODIL Models chat API (OpenAI-compatible: base_url
https://api.dodil.io/v1, bearer = an OIDC client-credentials token minted from the service account)
with MODEL. Ask for ONLY JSON: tickers (array), event_type (earnings|guidance|m_and_a|litigation|
insider|capital|macro|other), sentiment (bullish|bearish|neutral), materiality (high|med|low),
horizon (intraday|days|weeks), score (0-1), thesis (<=15 words). Retry on HTTP 429 with backoff;
• upserts {source_key: key, ...result} into `signals`.
5. List the filings/ and news/ object keys, shard them (~600 keys/shard), and fan out one Ignite
execution per shard.
6. Build an issuer/sector/supply-chain graph: integer-keyed node table (securities + sectors) + one edge
table with a `rel` column; populate it, CREATE GRAPH issuer_g, then show the 2-hop blast radius from a
high-conviction ticker joined to the signals table.
7. Show me today's high-materiality bullish signals ranked by score.
MODEL = kimi-k2 (fast + cheap for the bulk; swap to kimi-k2.6 for the sharpest labels — or cascade:
run kimi-k2 over everything, then re-run only the high-materiality items on kimi-k2.6.)Set MODEL once at the bottom — the single knob for the fast/quality trade-off. Merge-keying on
source_key means re-running is safe: a re-analyzed item refreshes its row instead of duplicating.
What made this work at scale
- Sourcing cost nothing — EDGAR is free and structured; storing 250k raw items in DataK3 objects is cheap.
- Analysis cost cents, not dollars — a quarter-million items on a low-cost model (
kimi-k2, escalating tokimi-k2.6), fanned out in parallel, with a desk-analyst read. The cascade spends the good model only on what's material. - Ignite is the batch engine — 0 → hundreds of concurrent executions, then back to zero: you pay for the 8-minute burst, not a standing cluster.
- Three pillars, one copy — SQL for the shortlist, a graph for the blast radius, vectors for meaning, all over the same rows — no glue, no second store, and any psql/Bolt client drops straight in.
- Re-analyzable forever — raw filings live in DataK3 objects, so a better prompt means re-running the batch, not re-sourcing the corpus.
Troubleshooting
- Not investment advice. This is a data pipeline; scores are model opinions. Backtest and add risk controls before anything touches real capital.
- A keyed upsert row vanished. A
nullor empty-string PRIMARY KEY reads back as null and drops the row. Use non-empty sentinels ("none","n/a") or omit the column and upsert with--merge. - New graph edges aren't traversed.
CREATE GRAPHsnapshots edges at creation. After adding edges,DROP GRAPH issuer_gand re-CREATEit. A v1 graph is one node + one edge table — relationship types live in therelcolumn and you filter them in the SQL join, not inside Cypher. - VECTOR upsert fails with "first frame too large". Send vector rows one per call — don't batch
many 2048-dim vectors into a single upsert. Embed with
jina-embeddings-v4(2048-dim). - Shard sizing. Balance shard size against per-execution time and the concurrency cap; too-large shards blunt the parallelism, too-small ones add overhead.
- Ticker mapping is noisy. Resolve entities with
gliner-multi-v2.1and keep them insecurities; keep atickers_jsonlist for multi-name items. - Data licensing. EDGAR is public; most news/market feeds are not — check terms before storing.
Ship it — make analyzer a public endpoint
The analyzer app above is shown as code. Ship it the last mile so users hit a real URL, not a snippet —
the managed runtime compiles your handler.py and returns a public FQDN:
Deploy the analyzer app publicly with no auth and give me its URL.
ignite_app_deploy→ignite_app_getDeployed analyzer (deployment_state: deployed). Its public FQDN on ignite.dodil.cloud is callable with no token — or call it with dodil ignite invoke analyzer.
dodil ignite app deploy analyzer --code ./analyzer --runtime python --allow-unauthenticated
dodil ignite app get analyzer --output json # → public_urlsThat's the quick path (managed compile). The full supply chain — DODIL git → CI checks → a scanned image in the DODIL registry → versioning and one-command rollback — is its own tutorial: Ship a DODIL App.
Conclusion
One DataK3 bucket that's the store from item #1 — a free corpus sourced as objects, a low-cost model for
the heavy analysis into a queryable signals table, an issuer/supply-chain graph that turns a single
signal into a blast radius, and vector search across every filing. A real trading-signal pipeline
that's low-cost, fast, and entirely on DODIL — the same warehouse → source → analyze → query backbone
as the leads warehouse, run at firehose scale.
Next steps:
- Leads Data Warehouse — the same pattern for sales.
- Customer-360 on DataK3 — entity resolution over the same three pillars.
- Inside the DODIL AI Cloud — the backbone agents offload to.