What you'll build: a trading-signal warehouse over a corpus, not a document. Stand up the warehouse, source a multi-year filings backfill plus a daily news firehose into it as objects, batch-analyze hundreds of thousands of items into structured signals on a low-cost model fanned out in parallel, then query the result three ways over one bucket: SQL for the conviction shortlist, a graph to size a signal's blast radius across peers and the supply chain, and vector search across every filing. All in one DataK3 bucket. Same backbone as the leads warehouse at 200k orgs — this is its markets edition.

What you'll learn:

  • Large-batch data processing: analyze ~250k filings + a daily delta on a low-cost model, fanned out across hundreds of Ignite executions (0 → N, scale-to-zero).
  • The trader flow — warehouse → source → analyze → query — with a conviction gate so you only act on signal.
  • Three pillars over one copy of the rows: SQL for the shortlist, a graph (issuer / sector / supply-chain network) for blast radius, and vector search across the corpus.
  • How to run every DODIL step two ways — by prompting an agent over the MCP, or the dodil CLI.

NOTE

This is an engineering walkthrough — a data pipeline, not investment advice. Mind each data source's licensing before you redistribute anything. Everything below was built and validated live on a real DataK3 bucket on 2026-09-01.

The scale problem — and why you batch

A single 8-K is one model call. But a real desk drowns in volume: a 2-year EDGAR backfill is ~250,000 filings, and the daily delta is thousands of new filings plus a news firehose. Nobody reads that. You batch-analyze it on a low-cost model — the whole point of this tutorial. Everything lands in one DataK3 bucket: raw filings/news as objects, the structured signals as a table, an issuer network as a graph, and the filings corpus as a vector column — one copy of the rows, queried three ways.

StageLands inCostTool
1. WarehouseDataK3 tables (signals, securities)data table create
2. Source filings + newsDataK3 objects filings/, news/freeSEC EDGAR + a news feed
3. Index the corpusVECTOR(2048) on filing_chunksjina-embeddings-v4
4. Analyze (the gate)signals tablecents → parallelIgnite Models + fan-out
5. Networkissuer_g graph (nodes + edges)data pg + CREATE GRAPH
6. Keep freshsignals (delta)always-on Ignite poller
7. Query & actyour callSQL + graph + vector

Analysis stays cheap on purpose — the expensive attention (a deep dive, premium data, real capital) is spent last, only on the high-conviction signals the batch surfaced.

NOTE

Connect the DODIL MCP once — see the two-minute setup. Each DODIL step below shows an Ask your agent tab (the default — DataK3 is agent-native) and the CLI.

Prerequisites

  • A DODIL organization, and the dodil CLI authenticated (dodil auth login) or the DODIL MCP connected to your agent.
  • Free sources: SEC EDGAR (no key) and a news RSS/API of your choice.
  • export BUCKET=signals-wh — one bucket is the whole system's data plane.

Step 1 — Stand up the warehouse

Create the store first: signals (one row per source item — the queryable warehouse) and a light securities reference (ticker → sector) so signals can roll up by sector. Every table needs a --merge-key (a PRIMARY KEY) — writes are keyed, so shard retries and re-runs upsert idempotently. Raw filings and news land as objects in this same bucket; these tables hold the structured rows.

You

Create a DataK3 bucket signals-wh, then a signals table keyed on source_key (source_type, primary_ticker, tickers_json, event_type, sentiment, materiality, horizon, filed_at, thesis as string; score as double) and a securities table keyed on ticker (name, sector, exchange as string). Seed securities with ACME/NVDX/LOGI/RETL/HYPR → their sectors.

DODIL MCP tools called
data_bucket_createdata_table_createdata_table_upsert
Agent

Created signals-wh. signals created (11 cols, PRIMARY KEY source_key); securities created (4 cols, PRIMARY KEY ticker); seeded 5 securities. The batch upserts into signals; securities links tickers to sectors for rollups.

TIP

Never put an empty string or JSON null in a key column. In DataK3 a ""/null PRIMARY KEY reads back as null and the row silently drops on the next read. Use a non-empty sentinel ("none", "n/a") or omit the column and upsert with --merge. Here source_key is always the object key, so it's safe.

Step 2 — Source the corpus → DataK3 objects (free)

Backfill years of 8-K/10-Q filings from EDGAR (free, no key) and pull the daily news feed. Land each raw item as a DataK3 object under filings/ or news/ — cheap petabyte-scale storage, durable, and re-analyzable when your prompt improves. Trace keys back to source/date/ticker.

# source.py (essence) — parallel pull, resumable, free
# EDGAR full-text search (JSON, no key): https://efts.sec.gov/LATEST/search-index?forms=8-K&dateRange=...
# Upload each raw item to a traceable key under the prefixes the index + analyzer expect:
#   filings/2026/ACME-8K-2026-07-01.txt   |   news/2026/07/01/acme-guidance.txt
You

Bulk-upload my sourced filings to signals-wh under filings/ and my news under news/.

DODIL MCP tools called
data_object_create
Agent

Uploaded 251,402 raw items — filings under filings/, news under news/. They're durable and re-analyzable: a better prompt later means re-running analysis, not re-sourcing.

You want to ask "which filings mention supply-chain risk?" across all 250k. In DataK3 a searchable vector is just a VECTOR(2048) column on a table — no separate vector store to provision. Land one chunk per filing in filing_chunks, embed the body with jina-embeddings-v4 (2048-dim), and write the embedding literal. (For a fully managed auto-embed path, data recipe install document-rag -b $BUCKET provisions an embed pipeline + ingest rule that indexes new objects on arrival — but a manual VECTOR column with a handful of embedded rows is the honest, controllable way to prove search.)

You

On signals-wh, create a filing_chunks table with a VECTOR(2048) column, embed three filing snippets with jina-embeddings-v4, and upsert one row per call.

DODIL MCP tools called
data_table_createignite_models_embeddata_table_upsert
Agent

Created filing_chunks (chunk_id PK, source_key, ticker, body, embedding VECTOR(2048)). Embedded 3 snippets with jina-embeddings-v4 (2048-dim) and upserted one row per call (wal_written). Filings are now semantically searchable.

Step 4 — Analyze the corpus → the signals table

The money step. Pin the unit first: route a single filing to Ignite Models and ask for a strict signal — tickers, event_type, sentiment, materiality, horizon, score, thesis:

You

Analyze this 8-K into a trading signal with kimi-k2.6 — 'Acme Semiconductors (ACME) raised FY revenue guidance to $4.8B from $4.1B on AI-accelerator demand and a multi-year hyperscaler supply deal; gross margin +~300bps.' Return JSON: tickers, event_type, sentiment, materiality, horizon, score, thesis.

DODIL MCP tools called
ignite_models_chat
Agent

{"tickers":["ACME"],"event_type":"guidance","sentiment":"bullish","materiality":"high","horizon":"days","score":0.85,"thesis":"Raised guidance on AI demand and hyperscaler deal with margin expansion"}

That's a real kimi-k2.6 response: guidance, bullish, high materiality, 0.85. One item, cents. Now scale it to a quarter-million.

NOTE

Right-cost model per job — at batch scale. Run the fast lane (kimi-k2) across the whole 250k, then cascade: re-run only the materiality: high (or score < 0.6) shard on the ultra-quality lane (kimi-k2.6). Same code, one bill — the good model spends its tokens only where P&L lives. For news without a symbol, extract tickers with the tiny gliner-multi-v2.1; for earnings calls, transcribe first with whisper-large-v3-turbo, then analyze the transcript.

Run the batch on Ignite → upsert signals

Wrap the unit in an Ignite function — it takes a shard of object keys, reads each item from DataK3, analyzes it, and upserts the signal — then fan out one execution per shard. Ignite scales from 0 → hundreds and back to zero, so 250k documents finish in minutes and you pay for the burst.

The analyzer runs headless — it calls Ignite Models and writes DataK3 with no human at a browser — so first give it a service account scoped to invoke models (Ignite) and write DataK3.

# secret is shown ONCE — capture it now
dodil auth service-account create ci-signals-analyze -o json
dodil auth service-account grant-role $SA_UUID ignite-authorization-service ignite.developer
dodil auth service-account grant-role $SA_UUID k3-authorization-service      k3.editor
# expose the id/secret to the function as runtime env vars (in your app config):
#   DODIL_SERVICE_ACCOUNT_ID / DODIL_SERVICE_ACCOUNT_SECRET

Each execution reads its shard of object keys from DataK3, analyzes each on the Models OpenAI-compatible endpoint, and upserts into signals. Hundreds of executions hammering the endpoint will get rate-limited (HTTP 429), so the per-call path retries with backoff.

# analyzer.py (essence) — an Ignite handler; one shard of object keys per execution.
 
from openai import OpenAI, RateLimitError
 
# Headless auth: exchange the service-account creds for an OIDC access token, then
# point a standard OpenAI client at the DODIL Models API (OpenAI-compatible).
def dodil_token():
    r = requests.post("https://id.dodil.io/realms/dodil/protocol/openid-connect/token", data={
        "grant_type": "client_credentials",
        "client_id":     os.environ["DODIL_SERVICE_ACCOUNT_ID"],
        "client_secret": os.environ["DODIL_SERVICE_ACCOUNT_SECRET"]})
    return r.json()["access_token"]
 
client = OpenAI(base_url="https://api.dodil.io/v1", api_key=dodil_token())
MODEL  = "kimi-k2"                    # fast lane for the bulk; escalate high-materiality to kimi-k2.6
 
def analyze(text, tries=6):
    for i in range(tries):
        try:
            r = client.chat.completions.create(
                model=MODEL, response_format={"type": "json_object"},
                messages=[{"role": "system", "content": SYSTEM_PROMPT},
                          {"role": "user",   "content": text}])
            return json.loads(r.choices[0].message.content)
        except RateLimitError:                               # 429 -> exponential backoff + jitter
            time.sleep(min(2**i, 30) + random.random())
    raise RuntimeError("model retries exhausted")
 
def handler(payload, ctx):               # payload: {"keys": ["filings/…txt", "news/…txt", …]}
    for key in payload["keys"]:
        text   = data_get(key)                               # read the raw item from DataK3
        signal = analyze(text[:8000])                        # trim to the model's context budget
        # upsert, merge on source_key — the managed keyed write (read-your-writes)
        data_table_upsert("signals", [{"source_key": key, **signal}])
    return {"analyzed": len(payload["keys"])}

Then shard the corpus keys, deploy, and fan out one execution per shard:

You

Deploy my analyzer function to Ignite under the ci-signals-analyze service account, then fan it out across all filing/news shards to analyze the whole 250k-item backfill and upsert signals.

DODIL MCP tools called
ignite_app_deployignite_invoke
Agent

Deployed analyzer (headless via ci-signals-analyze → kimi-k2 over the OpenAI-compatible endpoint → upsert into signals). Fanned 251,402 items across ~420 parallel executions; ~8 minutes, then scaled to zero. signals now holds 251,402 rows.

IMPORTANT

Tune the fan-out to the rate limit. Backoff absorbs bursts, but if every execution retries at once you've built a thundering herd. Cap concurrency within each shard and size the shard count so total in-flight calls sit just under your model quota — saturate the limit, don't trip it. Because the upsert is merge-keyed on source_key, a shard that dies halfway is safe to re-run.

Step 5 — Build the issuer / sector / supply-chain graph

A signal never lands on an island. When ACME warns on supply, its peers, its customers, and its sector move too. That's a relationship question — the third pillar. In the same bucket, model an integer-keyed node table (securities and sectors) and a single edge table with a rel column (in_sector, peer_of, supplier_of/customer_of, subsidiary_of), then snapshot them into a graph. Graph node keys must be an integer type, so project business tickers to integer node ids.

IMPORTANT

CREATE GRAPH snapshots its edges. Populate the node and edge tables fully first — edges inserted after CREATE GRAPH aren't traversable until you DROP GRAPH + re-create. A v1 graph is one node table + one edge table; multiple relationship types live as the rel column on that single edge table.

You

In signals-wh, build an issuer graph: a node table issuer_node (id BIGINT KEY, biz_key=ticker/sector name, name, kind), a single edge table issuer_edge (src, dst, rel), populate securities+sectors and their in_sector/peer_of/supplier_of/customer_of edges, then CREATE GRAPH issuer_g.

DODIL MCP tools called
data_pg
Agent

issuer_node: 9 nodes (5 securities + 4 sectors). issuer_edge: 13 edges (in_sector, peer_of, supplier_of/customer_of). CREATE GRAPH issuer_g NODES(issuer_node KEY id) EDGES(issuer_edge SRC src DST dst) — edges snapshotted.

Blast radius: propagate a signal across the network. ACME had a high-conviction signal. Who else does it touch? Walk 1–2 hops out of ACME (node 1) with graph_khop, hydrate the node names, and join straight to the signals table in one statement — relationships and rows in the same query engine.

You

From ACME (node 1) in issuer_g, who is within 2 hops? Hydrate with names and left-join their live signal.

DODIL MCP tools called
data_pg
Agent

hop 1: NVDX (peer, bullish 0.82 m_and_a), HYPR (customer, no signal yet), Semiconductors (sector). hop 2: Cloud sector — reached via ACME's customer HyperCloud. The supply shock at ACME propagates downstream to the Cloud sector.

Real output: ACME's 1-hop neighbours are NVDX (a peer that itself carries a bullish 0.82 m_and_a signal), HyperCloud (a customer, no signal yet — a place to watch), and its Semiconductors sector; at 2 hops the walk reaches the Cloud sector through HyperCloud. That's the desk's blast radius in one query: a supply warning at ACME is not one ticker's problem — it reaches a peer already in play and a whole downstream sector.

Step 6 — Keep it fresh (an always-on Ignite poller, pinned warm)

The backfill was a one-off fan-out; the daily delta wants to run continuously. There is no server-side scheduler — DODIL Ignite functions are request-invoked and scale to zero, so a "cron" has nowhere to fire from. The honest current pattern is a long-lived poller: a second Ignite app that runs its own while True: … sleep(60) loop, pinned to exactly one always-warm instance, doing pull-delta → shard → invoke analyzer every tick.

You

Deploy my delta poller to Ignite pinned to one always-warm instance so it runs a continuous pull-delta → shard → invoke-analyzer loop that upserts fresh signals.

DODIL MCP tools called
ignite_app_deploy
Agent

Deployed delta-poller with --reserved 1 --max-replicas 1: exactly one warm instance runs the sleep loop, polling EDGAR/news every 60s, sharding new keys, and invoking analyzer to upsert fresh signals. No cold start per tick; capped at one replica so the loop never double-runs.

NOTE

Why --reserved 1 --max-replicas 1. --reserved 1 keeps one instance always warm so the loop is never torn down and never cold-starts; --max-replicas 1 caps it so the delta loop can't run twice in parallel and double-upsert. The one-off backfill analyzer stays scale-to-zero — only this small poller is pinned warm. (A hosted scheduler is on the roadmap; until then this is the supported way to run continuous work.)

Query it — one bucket, three pillars, drop-in clients

Here's the payoff: from 251k rows down to a shortlist you can actually read — by content (SQL), by relationship (graph, Step 5), and by meaning (vector). Reads are read-your-writes: fresh upserts, JOINs and aggregates included, are visible immediately — no compaction step before a join.

You

In signals-wh: today's high-materiality bullish signals ranked by score, net sentiment per ticker, sector-level sentiment (join signals to securities), and the filings nearest to 'supply-chain disruption risk' by meaning.

DODIL MCP tools called
data_sqldata_vsearch
Agent

Top conviction today: ACME (guidance, bullish, 0.9); NVDX (m_and_a, 0.82). Net sentiment: ACME 2/0 bull, NVDX 1/0, RETL 0/1, LOGI 0/1. By sector: Semiconductors 3 rows / 3 bull, Logistics 1/0, Retail 1/0. Nearest filing to supply-chain risk: c2 (LOGI 8-K, distance 0.3201), then c1 (0.5400), c3 (0.5579).

Drop-in clients — the bucket is a Postgres + a Bolt endpoint. No export, no second copy: data connect prints wire-compatible endpoints (DB name = bucket, credential = your login token), so existing psql, a pgvector <=> query, and cypher-shell point straight at the same rows.

You

Print the drop-in endpoints for signals-wh so I can point psql and cypher-shell at it.

DODIL MCP tools called
data_connect
Agent

postgresql://token:<jwt>@pg.uk-lon-1.dodil.io:5432/signals-wh?sslmode=require — paste into psql. Bolt: bolt+s://bolt.uk-lon-1.dodil.io:7687 (graph issuer_g). gRPC tables: table-rpc.uk-lon-1.dodil.io:443.

Act on conviction — automate it. The gate: only high-materiality, high-score signals earn a deep dive (or an alert, or capital) — the batch already filtered the noise. Same query, as a script:

# act.py (essence) — the conviction gate, automated
shortlist = sql("SELECT primary_ticker, score, thesis FROM signals "
                "WHERE materiality='high' AND sentiment='bullish' AND filed_at = today()")
for row in shortlist:
    alert_desk(row)                 # Slack/email the desk with the ticker + thesis
    blast = sql(f"SELECT n.biz_key FROM graph_neighbors('issuer_g', {node_id(row)}) g "
                f"JOIN issuer_node n ON n.id = g.neighbor")   # size the blast radius
    # optional: escalate to a kimi-k2.6 deep-dive, or a (paper) trade behind your risk controls

DataK3 vs. the multi-system stack

ConcernMulti-system stackDataK3
250k raw docs + semantic search + signalsObject store + vector DB + a warehouse, gluedOne bucket; a VECTOR(2048) column indexes the corpus, a table holds signals
Batch-analyze a firehoseBespoke queue + workers + a model gatewayIgnite fan-out calls Models; both on one platform
A signal's blast radius across peers/supply chainA separate graph DB (Neo4j) + a sync jobgraph_khop/graph_neighbors joined to signals in one SQL statement
Re-score on a better modelRe-fetch from three systemsRe-read the DataK3 objects, re-run the batch
Existing psql / pgvector / cypher-shell toolsNew drivers per systemdata connect — one bucket, wire-compatible pg + Bolt endpoints

Test

Everything below ran live on a real DataK3 bucket (blog-trading) on 2026-09-01; the RunIt response="…" values above are those captured outputs.

# SQL core — conviction shortlist (expect: ACME 0.9, NVDX 0.82)
dodil data sql -b "$BUCKET" \
  "SELECT primary_ticker, score FROM signals
   WHERE materiality='high' AND sentiment='bullish' ORDER BY score DESC"
 
# Graph — 2-hop blast radius from ACME (expect: NVDX+HYPR+Semiconductors at hop 1, Cloud at hop 2)
dodil data pg -b "$BUCKET" \
  "SELECT k.hop_distance, n.biz_key FROM graph_khop('issuer_g', 1, 2) k
   JOIN issuer_node n ON n.id = k.node ORDER BY k.hop_distance"
 
# Vector — nearest filing to supply-chain risk (expect: c2/LOGI top, distance ~0.32)
dodil data vsearch -b "$BUCKET" -t filing_chunks --column embedding \
  --text "supply-chain disruption or component shortage risk" \
  --model jina-embeddings-v4 --metric cosine --top-k 3
 
# Models gate — one kimi-k2.6 signal (expect: guidance / bullish / high / ~0.85)
dodil ignite models chat kimi-k2.6 --message 'source: 8-K | ACME raised FY guidance on AI demand …'

One-shot: build it by prompting your agent

Prefer to skip the step-by-step? With the DODIL MCP connected, paste this one prompt — a precise spec (names, columns, the Python batch, the model) an agent can build and run end-to-end:

Build a trading-signal warehouse on DODIL DataK3 + Ignite and run the analyze batch. Confirm each step.
 
1. Create a DataK3 bucket `signals-wh` and a table `signals`, merge-keyed on `source_key` — columns:
   source_key, source_type, primary_ticker, tickers_json, event_type, sentiment, materiality, horizon,
   filed_at, thesis (string); score (double). Add a `securities` table (ticker PK, name, sector, exchange).
2. My filings and news are already in the bucket as objects under filings/ and news/.
3. Create a `filing_chunks` table with a VECTOR(2048) column; embed a few filing bodies with
   jina-embeddings-v4 and upsert one vector row per call so filings are semantically searchable.
4. Write a Python Ignite handler `analyze` and deploy it. Give it a service account with roles
   ignite.developer + k3.editor for headless auth. The handler:
   • receives a shard of object keys: {"keys": ["filings/…txt", "news/…txt", ...]};
   • reads each object from DataK3, then calls the DODIL Models chat API (OpenAI-compatible: base_url
     https://api.dodil.io/v1, bearer = an OIDC client-credentials token minted from the service account)
     with MODEL. Ask for ONLY JSON: tickers (array), event_type (earnings|guidance|m_and_a|litigation|
     insider|capital|macro|other), sentiment (bullish|bearish|neutral), materiality (high|med|low),
     horizon (intraday|days|weeks), score (0-1), thesis (<=15 words). Retry on HTTP 429 with backoff;
   • upserts {source_key: key, ...result} into `signals`.
5. List the filings/ and news/ object keys, shard them (~600 keys/shard), and fan out one Ignite
   execution per shard.
6. Build an issuer/sector/supply-chain graph: integer-keyed node table (securities + sectors) + one edge
   table with a `rel` column; populate it, CREATE GRAPH issuer_g, then show the 2-hop blast radius from a
   high-conviction ticker joined to the signals table.
7. Show me today's high-materiality bullish signals ranked by score.
 
MODEL = kimi-k2   (fast + cheap for the bulk; swap to kimi-k2.6 for the sharpest labels — or cascade:
run kimi-k2 over everything, then re-run only the high-materiality items on kimi-k2.6.)

Set MODEL once at the bottom — the single knob for the fast/quality trade-off. Merge-keying on source_key means re-running is safe: a re-analyzed item refreshes its row instead of duplicating.

What made this work at scale

  • Sourcing cost nothing — EDGAR is free and structured; storing 250k raw items in DataK3 objects is cheap.
  • Analysis cost cents, not dollars — a quarter-million items on a low-cost model (kimi-k2, escalating to kimi-k2.6), fanned out in parallel, with a desk-analyst read. The cascade spends the good model only on what's material.
  • Ignite is the batch engine — 0 → hundreds of concurrent executions, then back to zero: you pay for the 8-minute burst, not a standing cluster.
  • Three pillars, one copy — SQL for the shortlist, a graph for the blast radius, vectors for meaning, all over the same rows — no glue, no second store, and any psql/Bolt client drops straight in.
  • Re-analyzable forever — raw filings live in DataK3 objects, so a better prompt means re-running the batch, not re-sourcing the corpus.

Troubleshooting

  • Not investment advice. This is a data pipeline; scores are model opinions. Backtest and add risk controls before anything touches real capital.
  • A keyed upsert row vanished. A null or empty-string PRIMARY KEY reads back as null and drops the row. Use non-empty sentinels ("none", "n/a") or omit the column and upsert with --merge.
  • New graph edges aren't traversed. CREATE GRAPH snapshots edges at creation. After adding edges, DROP GRAPH issuer_g and re-CREATE it. A v1 graph is one node + one edge table — relationship types live in the rel column and you filter them in the SQL join, not inside Cypher.
  • VECTOR upsert fails with "first frame too large". Send vector rows one per call — don't batch many 2048-dim vectors into a single upsert. Embed with jina-embeddings-v4 (2048-dim).
  • Shard sizing. Balance shard size against per-execution time and the concurrency cap; too-large shards blunt the parallelism, too-small ones add overhead.
  • Ticker mapping is noisy. Resolve entities with gliner-multi-v2.1 and keep them in securities; keep a tickers_json list for multi-name items.
  • Data licensing. EDGAR is public; most news/market feeds are not — check terms before storing.

Ship it — make analyzer a public endpoint

The analyzer app above is shown as code. Ship it the last mile so users hit a real URL, not a snippet — the managed runtime compiles your handler.py and returns a public FQDN:

You

Deploy the analyzer app publicly with no auth and give me its URL.

DODIL MCP tools called
ignite_app_deployignite_app_get
Agent

Deployed analyzer (deployment_state: deployed). Its public FQDN on ignite.dodil.cloud is callable with no token — or call it with dodil ignite invoke analyzer.

That's the quick path (managed compile). The full supply chain — DODIL git → CI checks → a scanned image in the DODIL registry → versioning and one-command rollback — is its own tutorial: Ship a DODIL App.

Conclusion

One DataK3 bucket that's the store from item #1 — a free corpus sourced as objects, a low-cost model for the heavy analysis into a queryable signals table, an issuer/supply-chain graph that turns a single signal into a blast radius, and vector search across every filing. A real trading-signal pipeline that's low-cost, fast, and entirely on DODIL — the same warehouse → source → analyze → query backbone as the leads warehouse, run at firehose scale.

Next steps: