What you'll build: the front door of a CRM — the discover → qualify → nurture → convert pipeline —
on one DataK3 bucket. You discover companies into a discovery warehouse (SQL), qualify a bounded
batch on Ignite Models (kimi-k2.6) so you only pay to classify the leads you want, expand the target
list by vector lookalike (a VECTOR(2048) column), convert the strong buyers into leads +
opportunities (projecting a works_at edge into the account graph), and nurture them with an email
sequence that is data — walked by a scale-to-zero Ignite engine and tracked by a minimal public app.
One bucket answers by content (SQL), by meaning (vector), and by relationship (graph) over one copy of the
rows — no ETL, no Postgres + Pinecone + Neo4j + a warehouse to stitch.
What you'll learn:
- Stand up a discovery warehouse —
organizationsyou fill cheaply before you spend a credit. - Gate qualification on Ignite Models — classify a bounded batch to a buyer tier JSON verdict, cheaply.
- Expand the target list by meaning — vector-search
org_embeddingsfor lookalikes of your best buyers. - Convert strong buyers into
leads+opportunities— idempotently, with aconversionsaudit row. - Run email sequences as data — a
flow_stepsrow is an email; edit a campaign, don't redeploy. - Deploy a private tick engine + a public tracking split on Ignite, each with least-privilege identity.
The problem — and why it matters
Outbound is where a CRM bleeds money. The naive move is to enrich and email everyone, paying to qualify a list that is mostly noise. The fix is a cost gate: discover companies for free into a warehouse, classify only the bounded batch you choose on a cheap model, keep the strong buyers, and only then spend on conversion and sequencing. On a 200-company discovery run that's the difference between paying to process 200 and paying to process the ~20 that are real.
Everything lives in one DataK3 bucket. The crm/core masters (leads, contacts, accounts,
opportunities, activities) are the system of record; this skill owns the discovery warehouse and the
sequence engine and converts into those masters. The same rows answer three ways:
| Piece | Lands in | Pillar / runs on |
|---|---|---|
| Discovered companies | table organizations | SQL (the warehouse) |
| Qualify verdict | organizations.buyer / fit_score / angle | Ignite Models (kimi-k2.6, the cost gate) |
| Lookalike expansion | table org_embeddings (VECTOR(2048)) | Vector |
| Converted leads / opps | leads, opportunities, conversions (+ works_at in crm_graph) | SQL + Graph |
| Sequences as data | flows, flow_steps, flow_enrollments, flow_actions | SQL |
| The tick engine | writes sends → activities / flow_actions | Ignite app (private, scale-to-zero) |
| Open / click / unsubscribe | table email_events | Ignite app (public, minimal secrets) |
NOTE
Connect the DODIL MCP once — see the two-minute setup. Every step shows an Ask your agent tab (the default — DODIL is agent-native) and the CLI.
Prerequisites
- The
dodilCLI (dodil auth login) or the DODIL MCP connected to your agent (Claude Code, Cursor, …). export BUCKET=crm— one bucket is the whole system's data plane.- The
crm/coremasters in the bucket (leads,contacts,accounts,opportunities,activities). Building this skill standalone? Stub the ones this front door writes to — they carry over verbatim from Build a CRM on DataK3:
In the crm bucket, create the core master stubs this pipeline converts into: leads (key lead_id: email, org_domain, full_name, title, source, status, owner, score(double), created_at, updated_at), contacts (key email: full_name, org_domain, title, lifecycle_stage, source, owner, subscribed(boolean), created_at, updated_at), accounts (key org_domain: name, parent_domain, tier, country, industry), opportunities (key opportunity_id: name, account_domain, primary_contact_email, pipeline, stage, status, amount(double), owner, source, close_date, created_at), and activities (key activity_id: opportunity_id, contact_email, kind, subject, body, direction, status, ts).
data_bucket_create→data_table_createCreated bucket crm and the five crm/core master stubs (leads, contacts, accounts, opportunities, activities), all merge-keyed. In a full suite these already exist — crm/core owns them; this skill only writes into them.
export BUCKET=crm
dodil data bucket create "$BUCKET" --description "CRM — one bucket: discovery warehouse + sequence engine"
dodil data table create leads -b "$BUCKET" --merge-key lead_id \
--columns-json '[{"name":"lead_id","type":"string","nullable":false},{"name":"email","type":"string"},{"name":"org_domain","type":"string"},{"name":"full_name","type":"string"},{"name":"title","type":"string"},{"name":"source","type":"string"},{"name":"status","type":"string"},{"name":"owner","type":"string"},{"name":"score","type":"double"},{"name":"created_at","type":"string"},{"name":"updated_at","type":"string"}]'
dodil data table create contacts -b "$BUCKET" --merge-key email \
--columns-json '[{"name":"email","type":"string","nullable":false},{"name":"full_name","type":"string"},{"name":"org_domain","type":"string"},{"name":"title","type":"string"},{"name":"lifecycle_stage","type":"string"},{"name":"source","type":"string"},{"name":"owner","type":"string"},{"name":"subscribed","type":"boolean"},{"name":"created_at","type":"string"},{"name":"updated_at","type":"string"}]'
dodil data table create accounts -b "$BUCKET" --merge-key org_domain \
--columns-json '[{"name":"org_domain","type":"string","nullable":false},{"name":"name","type":"string"},{"name":"parent_domain","type":"string"},{"name":"tier","type":"string"},{"name":"country","type":"string"},{"name":"industry","type":"string"}]'
dodil data table create opportunities -b "$BUCKET" --merge-key opportunity_id \
--columns-json '[{"name":"opportunity_id","type":"string","nullable":false},{"name":"name","type":"string"},{"name":"account_domain","type":"string"},{"name":"primary_contact_email","type":"string"},{"name":"pipeline","type":"string"},{"name":"stage","type":"string"},{"name":"status","type":"string"},{"name":"amount","type":"double"},{"name":"owner","type":"string"},{"name":"source","type":"string"},{"name":"close_date","type":"string"},{"name":"created_at","type":"string"}]'
dodil data table create activities -b "$BUCKET" --merge-key activity_id \
--columns-json '[{"name":"activity_id","type":"string","nullable":false},{"name":"opportunity_id","type":"string"},{"name":"contact_email","type":"string"},{"name":"kind","type":"string"},{"name":"subject","type":"string"},{"name":"body","type":"string"},{"name":"direction","type":"string"},{"name":"status","type":"string"},{"name":"ts","type":"string"}]'# models.py — the crm/core masters this front door converts into (natural PKs, never SERIAL).
# crm/core OWNS these; they're stubbed minimally here so the package runs standalone. Money is
# DECIMAL (opportunities.amount); the 0-1 scores are plain floats.
from datetime import datetime
from decimal import Decimal
from pgvector.sqlalchemy import Vector
from sqlalchemy import BigInteger, Boolean, DateTime, Float, Integer, Numeric, String
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column
class Base(DeclarativeBase):
pass
class Lead(Base):
__tablename__ = "leads"
lead_id: Mapped[str] = mapped_column(String, primary_key=True) # e.g. "lead:<domain>"
email: Mapped[str | None] = mapped_column(String, nullable=True)
org_domain: Mapped[str | None] = mapped_column(String, nullable=True)
full_name: Mapped[str | None] = mapped_column(String, nullable=True)
title: Mapped[str | None] = mapped_column(String, nullable=True)
source: Mapped[str | None] = mapped_column(String, nullable=True)
status: Mapped[str | None] = mapped_column(String, nullable=True)
owner: Mapped[str | None] = mapped_column(String, nullable=True)
score: Mapped[float | None] = mapped_column(Float, nullable=True) # = fit_score, 0-1
created_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)
updated_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)
class Contact(Base):
__tablename__ = "contacts"
email: Mapped[str] = mapped_column(String, primary_key=True) # natural key
full_name: Mapped[str | None] = mapped_column(String, nullable=True)
org_domain: Mapped[str | None] = mapped_column(String, nullable=True)
title: Mapped[str | None] = mapped_column(String, nullable=True)
lifecycle_stage: Mapped[str | None] = mapped_column(String, nullable=True)
source: Mapped[str | None] = mapped_column(String, nullable=True)
owner: Mapped[str | None] = mapped_column(String, nullable=True)
subscribed: Mapped[bool | None] = mapped_column(Boolean, nullable=True)
created_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)
updated_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)
class Account(Base):
__tablename__ = "accounts"
org_domain: Mapped[str] = mapped_column(String, primary_key=True) # natural key (the join key)
name: Mapped[str | None] = mapped_column(String, nullable=True)
parent_domain: Mapped[str | None] = mapped_column(String, nullable=True)
tier: Mapped[str | None] = mapped_column(String, nullable=True)
country: Mapped[str | None] = mapped_column(String, nullable=True)
industry: Mapped[str | None] = mapped_column(String, nullable=True)
class Opportunity(Base):
__tablename__ = "opportunities"
opportunity_id: Mapped[str] = mapped_column(String, primary_key=True) # e.g. "opp:<domain>"
name: Mapped[str | None] = mapped_column(String, nullable=True)
account_domain: Mapped[str | None] = mapped_column(String, nullable=True)
primary_contact_email: Mapped[str | None] = mapped_column(String, nullable=True)
pipeline: Mapped[str | None] = mapped_column(String, nullable=True)
stage: Mapped[str | None] = mapped_column(String, nullable=True)
status: Mapped[str | None] = mapped_column(String, nullable=True)
amount: Mapped[Decimal | None] = mapped_column(Numeric(18, 2), nullable=True) # money -> DECIMAL
owner: Mapped[str | None] = mapped_column(String, nullable=True)
source: Mapped[str | None] = mapped_column(String, nullable=True)
close_date: Mapped[str | None] = mapped_column(String, nullable=True)
created_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)
class Activity(Base):
__tablename__ = "activities"
activity_id: Mapped[str] = mapped_column(String, primary_key=True)
opportunity_id: Mapped[str | None] = mapped_column(String, nullable=True)
contact_email: Mapped[str | None] = mapped_column(String, nullable=True)
kind: Mapped[str | None] = mapped_column(String, nullable=True)
subject: Mapped[str | None] = mapped_column(String, nullable=True)
body: Mapped[str | None] = mapped_column(String, nullable=True)
direction: Mapped[str | None] = mapped_column(String, nullable=True)
status: Mapped[str | None] = mapped_column(String, nullable=True)
ts: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)Step 1 — Stand up the discovery warehouse
Before you spend a credit, you fill a cheap table. organizations (key domain) is the warehouse: a row per
discovered company, with the fields a gate needs to judge fit (category, ai_native, data_intensity,
description) and the columns the gate will write back (buyer, fit_score, angle). org_embeddings
(key domain, a VECTOR(2048) column) is its vector twin — one embedding per company, for dedup and
lookalike expansion. A --merge-key (PRIMARY KEY) is required: re-running discovery upserts idempotently
instead of duplicating.
In crm, create the discovery warehouse: organizations (key domain) with organization, country, headcount_band, layer, category, ai_native, data_intensity, buyer, angle, description, fit_score(double), node_id(long), emails_count(int), verified(boolean); and org_embeddings (key domain) with organization and an embedding VECTOR(2048), with an index on the vector column.
data_table_create→data_pgCreated organizations (key domain, 15 columns) and org_embeddings (key domain) with an indexed VECTOR(2048) column. Discovered rows land in organizations first; the gate writes buyer/fit_score/angle back onto the same row.
dodil data table create organizations -b "$BUCKET" --merge-key domain \
--columns-json '[
{"name":"domain","type":"string","nullable":false},
{"name":"organization","type":"string"},
{"name":"country","type":"string"},
{"name":"headcount_band","type":"string"},
{"name":"layer","type":"string"},
{"name":"category","type":"string"},
{"name":"ai_native","type":"string"},
{"name":"data_intensity","type":"string"},
{"name":"buyer","type":"string"},
{"name":"angle","type":"string"},
{"name":"description","type":"string"},
{"name":"fit_score","type":"double"},
{"name":"node_id","type":"long"},
{"name":"emails_count","type":"int"},
{"name":"verified","type":"boolean"}
]'
dodil data table create org_embeddings -b "$BUCKET" --merge-key domain \
--columns-json '[
{"name":"domain","type":"string","nullable":false},
{"name":"organization","type":"string"},
{"name":"embedding","type":"VECTOR(2048)"}
]'
# index the vector column — an unindexed VECTOR is an exact scan on every KNN.
# (All four of the suite's vector tables carry this index.)
dodil data pg -b "$BUCKET" "CREATE INDEX ON org_embeddings (embedding)"# models.py — the discovery warehouse this skill OWNS. The gate (routes.qualify) writes
# buyer/fit_score/angle/verified BACK onto the same organizations row; org_embeddings is its
# VECTOR(2048) twin, keyed on the same domain, for lookalike KNN (pgvector `<=>`).
class Organization(Base):
__tablename__ = "organizations"
domain: Mapped[str] = mapped_column(String, primary_key=True) # natural key
organization: Mapped[str | None] = mapped_column(String, nullable=True)
country: Mapped[str | None] = mapped_column(String, nullable=True)
headcount_band: Mapped[str | None] = mapped_column(String, nullable=True)
layer: Mapped[str | None] = mapped_column(String, nullable=True)
category: Mapped[str | None] = mapped_column(String, nullable=True)
ai_native: Mapped[str | None] = mapped_column(String, nullable=True)
data_intensity: Mapped[str | None] = mapped_column(String, nullable=True)
buyer: Mapped[str | None] = mapped_column(String, nullable=True) # unqualified|strong|possible|weak
angle: Mapped[str | None] = mapped_column(String, nullable=True)
description: Mapped[str | None] = mapped_column(String, nullable=True)
fit_score: Mapped[float | None] = mapped_column(Float, nullable=True) # 0-1, not money -> Float
node_id: Mapped[int | None] = mapped_column(BigInteger, nullable=True) # graph node id (long)
emails_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
verified: Mapped[bool | None] = mapped_column(Boolean, nullable=True)
class OrgEmbedding(Base):
__tablename__ = "org_embeddings"
domain: Mapped[str] = mapped_column(String, primary_key=True)
organization: Mapped[str | None] = mapped_column(String, nullable=True)
embedding = mapped_column(Vector(2048), nullable=True)Fill it. Discovery is source-agnostic — a crawler, a data provider, a CSV — anything that emits
{domain, description}. Each is an upsert; buyer starts at the sentinel "unqualified" (never "" — an
empty string in a written column reads back fine, but keeping a real sentinel makes the "not yet judged"
filter honest):
In crm, upsert four discovered companies into organizations, all buyer=unqualified for now: Harborlytics (harborlytics.io, a customer data platform stitching Postgres, Pinecone and a warehouse), Greyparrot (greyparrot.ai, AI waste analytics with computer vision), NimbusGraph (nimbusgraph.dev, a knowledge-graph + semantic-retrieval dev platform), and BrightMugs (brightmugs.com, a DTC merch store on off-the-shelf ecommerce SaaS).
data_table_upsertUpserted 4 organizations (wal_written: true), all buyer=unqualified — the warehouse is filled and free; nothing has cost a credit yet.
dodil data table upsert organizations -b "$BUCKET" \
--row '{"domain":"harborlytics.io","organization":"Harborlytics","country":"US","headcount_band":"201-500","layer":"platform","category":"customer data platform","ai_native":"yes","data_intensity":"high","buyer":"unqualified","angle":"none","description":"Customer data platform unifying events, profiles, and vector similarity for real-time personalization; currently stitches Postgres, Pinecone, and a warehouse.","fit_score":0.0,"node_id":0,"emails_count":0,"verified":false}'
dodil data table upsert organizations -b "$BUCKET" \
--row '{"domain":"greyparrot.ai","organization":"Greyparrot","country":"UK","headcount_band":"51-200","layer":"application","category":"AI waste analytics","ai_native":"yes","data_intensity":"high","buyer":"unqualified","angle":"none","description":"AI waste analytics; computer vision on sorting facilities producing per-object composition data at scale.","fit_score":0.0,"node_id":0,"emails_count":0,"verified":false}'
dodil data table upsert organizations -b "$BUCKET" \
--row '{"domain":"nimbusgraph.dev","organization":"NimbusGraph","country":"DE","headcount_band":"11-50","layer":"platform","category":"knowledge graph infra","ai_native":"yes","data_intensity":"high","buyer":"unqualified","angle":"none","description":"Developer platform for knowledge graphs and semantic retrieval over enterprise documents; needs graph plus vector plus SQL in one place.","fit_score":0.0,"node_id":0,"emails_count":0,"verified":false}'
dodil data table upsert organizations -b "$BUCKET" \
--row '{"domain":"brightmugs.com","organization":"BrightMugs","country":"US","headcount_band":"11-50","layer":"application","category":"ecommerce merch","ai_native":"no","data_intensity":"low","buyer":"unqualified","angle":"none","description":"Direct-to-consumer printed mugs and merchandise store running on an off-the-shelf ecommerce SaaS.","fit_score":0.0,"node_id":0,"emails_count":0,"verified":false}'TIP
intake is a knob. This is the discovery path (default). Running manual instead? Skip the
warehouse entirely and hand-upsert leads directly — Steps 1–3 fall away and you start at Step 4.
Step 2 — Qualify a bounded batch on the cost gate (Ignite Models)
Here is the money. You classify only the batch you ask for (qualify_batch, default 25) on
kimi-k2.6, and the model returns a buyer tier verdict as JSON. The system prompt is rendered from
product_pitch — what counts as a fit is the knob that decides your whole funnel. Only the tier you keep
(auto_convert_tier, default strong) will cost you anything downstream.
Qualify this discovered company as a sales lead for a unified data backend (SQL+vector+graph in one bucket), on kimi-k2.6. Return ONLY JSON: buyer (strong|possible|weak), fit_score (0-1), angle (<=12 words). Company — org: Harborlytics | domain: harborlytics.io | Customer data platform unifying events, profiles, and vector similarity for real-time personalization; currently stitches Postgres, Pinecone, and a warehouse.
ignite_models_chat{"buyer": "strong", "fit_score": 0.92, "angle": "Replace stitched Postgres/Pinecone/warehouse with unified backend"}
# GATE — classify a bounded batch (qualify_batch=25) on kimi-k2.6. One call per org shown; loop your batch.
dodil ignite models chat kimi-k2.6 \
--system 'Qualify a company as a sales lead for a unified data backend (SQL+vector+graph in one bucket). Return ONLY JSON: buyer (strong|possible|weak), fit_score (0-1), angle (<=12 words).' \
--message 'org: Harborlytics | domain: harborlytics.io | Customer data platform unifying events, profiles, and vector similarity for real-time personalization; currently stitches Postgres, Pinecone, and a warehouse.'
# -> {"buyer": "strong", "fit_score": 0.92, "angle": "Replace stitched Postgres/Pinecone/warehouse with unified backend"}Write the verdict back onto the same row — a partial --merge upsert touches only buyer, fit_score,
angle, verified, leaving the discovery fields intact. (Live batch on 2026-09-01: Harborlytics strong
0.92, Greyparrot strong 0.90, NimbusGraph strong 0.98, BrightMugs weak 0.10 — the DTC merch store is
correctly rejected.)
In crm, write the qualify verdicts back onto organizations with a partial merge (only buyer, fit_score, angle, verified): harborlytics.io strong/0.92, greyparrot.ai strong/0.90, nimbusgraph.dev strong/0.98, brightmugs.com weak/0.10, each with its angle and verified=true.
data_table_upsertMerged the four verdicts onto organizations (wal_written: true) — buyer/fit_score/angle set, discovery fields untouched. Three strong, one weak. Only the strong tier moves on.
dodil data table upsert organizations -b "$BUCKET" --merge \
--row '{"domain":"harborlytics.io","buyer":"strong","fit_score":0.92,"angle":"Replace stitched Postgres/Pinecone/warehouse with unified backend","verified":true}'
dodil data table upsert organizations -b "$BUCKET" --merge \
--row '{"domain":"greyparrot.ai","buyer":"strong","fit_score":0.90,"angle":"Unify object analytics, vision embeddings, and material relationships","verified":true}'
dodil data table upsert organizations -b "$BUCKET" --merge \
--row '{"domain":"nimbusgraph.dev","buyer":"strong","fit_score":0.98,"angle":"One platform solves their graph, vector, SQL fragmentation","verified":true}'
dodil data table upsert organizations -b "$BUCKET" --merge \
--row '{"domain":"brightmugs.com","buyer":"weak","fit_score":0.10,"angle":"SaaS DTC store lacks engineering need for unified backend","verified":true}'Step 3 — Expand the target list by lookalike (Vector)
You've qualified a batch and found strong buyers. Now find more like them without a second gate run. Embed
each company's description into org_embeddings with jina-embeddings-v4, and a KNN search returns the
companies most similar in meaning to your best buyers — a self-service lookalike audience over one copy of
the rows, no separate vector store.
In crm, embed each organization's description with jina-embeddings-v4 and upsert one row per company into org_embeddings (one vector per upsert call).
ignite_models_embed→data_table_upsertEmbedded 4 descriptions with jina-embeddings-v4 (2048-dim) and upserted them one row per call into org_embeddings — batching many 2048-dim vectors in one upsert can hit a gRPC frame limit, so it's one vector per write.
# embed a description, then upsert it as a [f1,f2,…] literal — ONE vector row per upsert call
DESC="Customer data platform unifying events, profiles, and vector similarity for real-time personalization; currently stitches Postgres, Pinecone, and a warehouse."
VEC=$(dodil ignite models embed jina-embeddings-v4 --input "$DESC" -o json \
| python3 -c "import sys,json; print(json.dumps(json.load(sys.stdin)['data']['data'][0]['embedding']))")
dodil data table upsert org_embeddings -b "$BUCKET" \
--row "{\"domain\":\"harborlytics.io\",\"organization\":\"Harborlytics\",\"embedding\":$VEC}"
# … repeat for greyparrot.ai, nimbusgraph.dev, brightmugs.comNow expand: search org_embeddings for lookalikes of your strong-buyer profile. The three data-infra
companies rank ahead of the DTC merch store (which sits furthest, cosine 0.56) — a target list you grew for
the price of one embedding:
In crm, find lookalikes of our strong buyers: vector-search org_embeddings with the profile 'fragmented data stack stitching a SQL database, a vector store, and a graph database for AI retrieval' and return the nearest companies by cosine.
data_vsearchNearest: harborlytics.io (0.33), nimbusgraph.dev (0.39), greyparrot.ai (0.47) — the three data-infra strong buyers — then brightmugs.com (0.56), the weak DTC store, correctly last. The lookalike audience is the top of that list.
dodil data vsearch -b "$BUCKET" -t org_embeddings --column embedding \
--text "fragmented data stack stitching a SQL database, a vector store, and a graph database for AI retrieval" \
--model jina-embeddings-v4 --metric cosine --top-k 4
# id score
# harborlytics.io 0.3331 <- nearest
# nimbusgraph.dev 0.3885
# greyparrot.ai 0.4748
# brightmugs.com 0.5601 <- weak DTC store, correctly furthestStep 4 — Convert strong buyers into leads + opportunities
auto_convert_tier=strong means only buyer='strong' rows convert. For each, you write a lead, and — for
the ones you work — an account, a contact, an opportunity, and a conversions audit row that
ties domain → lead → opportunity → contact together. Every write is a keyed upsert, so re-running the
conversion is idempotent — one conversions row per domain, no matter how many times the pipeline retries.
In crm, convert the strong organizations into leads (status qualified, source discovery, score = fit_score, lead_id 'lead:'+domain). Then fully convert harborlytics.io: upsert an account (Harborlytics, tier mid, US, software), a contact ([email protected], Head of Data, lifecycle lead, subscribed true), an opportunity (opp:harborlytics.io, pipeline sales, stage discovery, status open), and a conversions row conv:harborlytics.io linking domain→lead→opportunity→contact.
data_table_upsertWrote 3 qualified leads (harborlytics.io, greyparrot.ai, nimbusgraph.dev). Fully converted harborlytics.io: account + contact + opportunity opp:harborlytics.io + conversion conv:harborlytics.io. Re-running writes the same keys — one conversions row per domain.
# strong tier -> leads (one lead per strong org; keyed lead:<domain> so re-runs upsert)
dodil data table upsert leads -b "$BUCKET" \
--row '{"lead_id":"lead:harborlytics.io","email":"[email protected]","org_domain":"harborlytics.io","full_name":"Harborlytics","title":"discovery","source":"discovery","status":"qualified","owner":"[email protected]","score":0.92,"created_at":"2026-09-01T12:00:00Z","updated_at":"2026-09-01T12:00:00Z"}'
dodil data table upsert leads -b "$BUCKET" \
--row '{"lead_id":"lead:greyparrot.ai","email":"[email protected]","org_domain":"greyparrot.ai","full_name":"Greyparrot","title":"discovery","source":"discovery","status":"qualified","owner":"[email protected]","score":0.90,"created_at":"2026-09-01T12:00:00Z","updated_at":"2026-09-01T12:00:00Z"}'
dodil data table upsert leads -b "$BUCKET" \
--row '{"lead_id":"lead:nimbusgraph.dev","email":"[email protected]","org_domain":"nimbusgraph.dev","full_name":"NimbusGraph","title":"discovery","source":"discovery","status":"qualified","owner":"[email protected]","score":0.98,"created_at":"2026-09-01T12:00:00Z","updated_at":"2026-09-01T12:00:00Z"}'
# work the top one: account + contact + opportunity + the conversions audit row
dodil data table upsert accounts -b "$BUCKET" \
--row '{"org_domain":"harborlytics.io","name":"Harborlytics","parent_domain":"none","tier":"mid","country":"US","industry":"software"}'
dodil data table upsert contacts -b "$BUCKET" \
--row '{"email":"[email protected]","full_name":"Harborlytics Data Team","org_domain":"harborlytics.io","title":"Head of Data","lifecycle_stage":"lead","source":"discovery","owner":"[email protected]","subscribed":true,"created_at":"2026-09-01T12:00:00Z","updated_at":"2026-09-01T12:00:00Z"}'
dodil data table upsert opportunities -b "$BUCKET" \
--row '{"opportunity_id":"opp:harborlytics.io","name":"Harborlytics — unified backend","account_domain":"harborlytics.io","primary_contact_email":"[email protected]","pipeline":"sales","stage":"discovery","status":"open","amount":0.0,"owner":"[email protected]","source":"discovery","close_date":"none","created_at":"2026-09-01T12:00:00Z"}'
dodil data table upsert conversions -b "$BUCKET" \
--row '{"conversion_id":"conv:harborlytics.io","domain":"harborlytics.io","lead_id":"lead:harborlytics.io","opportunity_id":"opp:harborlytics.io","contact_email":"[email protected]","converted_at":"2026-09-01T12:00:00Z","owner":"[email protected]"}'A converted company also joins the account graph so family pipeline rollups (from crm/core) include it.
The core graph is table-backed — an integer-keyed crm_node, a crm_edge with a rel column. On conversion
you project the new account (node id in the account range 1–9,999) and contact (10,000–99,999) and a
works_at edge, then — because CREATE GRAPH snapshots its edges — populate the tables first and create
the graph last. (In the suite, crm/account-360 owns the single deferred assembly; standalone, this front
door creates it.)
In crm, project the converted Harborlytics account and contact into crm_node (account id 1, contact id 10001) and a works_at edge contact->account into crm_edge, then CREATE GRAPH crm_graph over them. From the account (node 1) traverse inward one hop to confirm the works_at edge reaches the contact.
data_pgcrm_node has the account (1) and contact (10001); crm_edge has works_at (10001->1). CREATE GRAPH crm_graph snapshotted the edge. graph_khop('crm_graph',1,1,'in') returns the contact 'Harborlytics Data Team' — the converted org is in the graph.
# integer-keyed node + edge tables (crm/core owns these; shown here for a standalone build)
dodil data pg -b "$BUCKET" "CREATE TABLE crm_node (id BIGINT PRIMARY KEY, kind VARCHAR, biz_key VARCHAR, name VARCHAR)"
dodil data pg -b "$BUCKET" "CREATE TABLE crm_edge (src BIGINT, dst BIGINT, rel VARCHAR, PRIMARY KEY (src,dst))"
dodil data pg -b "$BUCKET" "INSERT INTO crm_node VALUES
(1,'account','harborlytics.io','Harborlytics'),
(10001,'contact','[email protected]','Harborlytics Data Team')"
dodil data pg -b "$BUCKET" "INSERT INTO crm_edge VALUES (10001,1,'works_at')"
# populate FIRST, then snapshot the graph
dodil data pg -b "$BUCKET" "CREATE GRAPH crm_graph NODES (crm_node KEY id) EDGES (crm_edge SRC src DST dst)"
# reverse one-hop from the account finds the contact who works_at it
dodil data pg -b "$BUCKET" "
SELECT c.name AS contact, c.biz_key AS email
FROM graph_khop('crm_graph', 1, 1, 'in') g
JOIN crm_node c ON c.id = g.node AND c.kind='contact'"
# contact | email
# Harborlytics Data Team | [email protected]Step 5 — Model the sequence engine as data
nurture_flow=true turns the pipeline into a drip. A sequence is four tables — flows is the campaign,
flow_steps are the emails, flow_enrollments track each contact's position, flow_actions is the audit
log. The point: a campaign is rows you edit, not code you ship. Add an email → upsert a flow_steps row;
reorder → edit position; pause → flip flows.status. No redeploy either way.
In crm, create flows (key flow_id: name, trigger_event, status, created_at), flow_steps (key step_id: flow_id, position(int), kind, subject, body_html, delay_seconds(int), stage), flow_enrollments (key enrollment_id: flow_id, contact_email, opportunity_id, status, current_position(int), next_run_at, enrolled_at), and flow_actions (key action_id: enrollment_id, flow_id, step_id, contact_email, kind, status, detail, ts).
data_table_createCreated flows, flow_steps, flow_enrollments, flow_actions — all merge-keyed. Authoring a campaign is now an upsert into flow_steps; the engine reads these to know what to send and when.
dodil data table create flows -b "$BUCKET" --merge-key flow_id \
--columns-json '[{"name":"flow_id","type":"string","nullable":false},{"name":"name","type":"string"},{"name":"trigger_event","type":"string"},{"name":"status","type":"string"},{"name":"created_at","type":"string"}]'
dodil data table create flow_steps -b "$BUCKET" --merge-key step_id \
--columns-json '[{"name":"step_id","type":"string","nullable":false},{"name":"flow_id","type":"string","nullable":false},{"name":"position","type":"int","nullable":false},{"name":"kind","type":"string"},{"name":"subject","type":"string"},{"name":"body_html","type":"string"},{"name":"delay_seconds","type":"int"},{"name":"stage","type":"string"}]'
dodil data table create flow_enrollments -b "$BUCKET" --merge-key enrollment_id \
--columns-json '[{"name":"enrollment_id","type":"string","nullable":false},{"name":"flow_id","type":"string"},{"name":"contact_email","type":"string"},{"name":"opportunity_id","type":"string"},{"name":"status","type":"string"},{"name":"current_position","type":"int"},{"name":"next_run_at","type":"string"},{"name":"enrolled_at","type":"string"}]'
dodil data table create flow_actions -b "$BUCKET" --merge-key action_id \
--columns-json '[{"name":"action_id","type":"string","nullable":false},{"name":"enrollment_id","type":"string"},{"name":"flow_id","type":"string"},{"name":"step_id","type":"string"},{"name":"contact_email","type":"string"},{"name":"kind","type":"string"},{"name":"status","type":"string"},{"name":"detail","type":"string"},{"name":"ts","type":"string"}]'# models.py — the sequence engine as data: a campaign is rows you edit, not code you ship.
# Add an email -> upsert a FlowStep; reorder -> edit position; pause -> flip Flow.status.
class Flow(Base):
__tablename__ = "flows"
flow_id: Mapped[str] = mapped_column(String, primary_key=True)
name: Mapped[str | None] = mapped_column(String, nullable=True)
trigger_event: Mapped[str | None] = mapped_column(String, nullable=True)
status: Mapped[str | None] = mapped_column(String, nullable=True)
created_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)
class FlowStep(Base):
__tablename__ = "flow_steps"
step_id: Mapped[str] = mapped_column(String, primary_key=True)
flow_id: Mapped[str | None] = mapped_column(String, nullable=True)
position: Mapped[int | None] = mapped_column(Integer, nullable=True)
kind: Mapped[str | None] = mapped_column(String, nullable=True)
subject: Mapped[str | None] = mapped_column(String, nullable=True)
body_html: Mapped[str | None] = mapped_column(String, nullable=True)
delay_seconds: Mapped[int | None] = mapped_column(Integer, nullable=True)
stage: Mapped[str | None] = mapped_column(String, nullable=True)
class FlowEnrollment(Base):
__tablename__ = "flow_enrollments"
enrollment_id: Mapped[str] = mapped_column(String, primary_key=True)
flow_id: Mapped[str | None] = mapped_column(String, nullable=True)
contact_email: Mapped[str | None] = mapped_column(String, nullable=True)
opportunity_id: Mapped[str | None] = mapped_column(String, nullable=True)
status: Mapped[str | None] = mapped_column(String, nullable=True)
current_position: Mapped[int | None] = mapped_column(Integer, nullable=True)
next_run_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)
enrolled_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)
class FlowAction(Base):
__tablename__ = "flow_actions"
action_id: Mapped[str] = mapped_column(String, primary_key=True)
enrollment_id: Mapped[str | None] = mapped_column(String, nullable=True)
flow_id: Mapped[str | None] = mapped_column(String, nullable=True)
step_id: Mapped[str | None] = mapped_column(String, nullable=True)
contact_email: Mapped[str | None] = mapped_column(String, nullable=True)
kind: Mapped[str | None] = mapped_column(String, nullable=True)
status: Mapped[str | None] = mapped_column(String, nullable=True)
detail: Mapped[str | None] = mapped_column(String, nullable=True)
ts: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)Author the campaign and enroll your converted contact. Each email is one row; delay_seconds is when it
fires after the previous step (0 / 3 days / 7 days). The enrollment starts at current_position 1,
next_run_at now — the engine picks it up on its next tick.
In crm, create an onboarding flow (trigger lead_converted, status active) with three email steps at positions 1/2/3 and delays 0 / 259200 / 604800 seconds. Then enroll [email protected] (opportunity opp:harborlytics.io) at current_position 1, next_run_at now.
data_table_upsertInserted the onboarding flow, three email steps (delays 0 / 3d / 7d), and enrollment enr:harborlytics.io at current_position 1, active, due now. The engine sends step 1 on its next tick and schedules step 2 for +3 days.
dodil data table upsert flows -b "$BUCKET" \
--row '{"flow_id":"onboarding","name":"Onboarding","trigger_event":"lead_converted","status":"active","created_at":"2026-09-01T12:00:00Z"}'
dodil data table upsert flow_steps -b "$BUCKET" --row '{"step_id":"onb-1","flow_id":"onboarding","position":1,"kind":"email","subject":"One backend for SQL, vector, and graph","body_html":"<p>Your data, one bucket — no ETL between three systems.</p>","delay_seconds":0,"stage":"discovery"}'
dodil data table upsert flow_steps -b "$BUCKET" --row '{"step_id":"onb-2","flow_id":"onboarding","position":2,"kind":"email","subject":"Retire the Postgres + Pinecone + warehouse stitch","body_html":"<p>See how one bucket answers by content, meaning, and relationship.</p>","delay_seconds":259200,"stage":"demo"}'
dodil data table upsert flow_steps -b "$BUCKET" --row '{"step_id":"onb-3","flow_id":"onboarding","position":3,"kind":"email","subject":"Book a 30-minute unified-backend walkthrough","body_html":"<p>Bring your schema — we will model it live on DataK3.</p>","delay_seconds":604800,"stage":"proposal"}'
dodil data table upsert flow_enrollments -b "$BUCKET" \
--row '{"enrollment_id":"enr:harborlytics.io","flow_id":"onboarding","contact_email":"[email protected]","opportunity_id":"opp:harborlytics.io","status":"active","current_position":1,"next_run_at":"2026-09-01T12:00:00Z","enrolled_at":"2026-09-01T12:00:00Z"}'Routes
The download (see Get the code) fronts the warehouse with a small FastAPI app,
routes.py — CRUD over the discovery warehouse plus the three ops that turn discovered
companies into pipeline: a cost-gated qualify, a lookalike expand, and a convert.
This is the app layer of Steps 1–4. The routes live on an APIRouter — the suite app mounts
all seven CRM components on one FastAPI under per-component prefixes (this one at
/lead-to-opportunity) — while app = FastAPI(...) at the bottom keeps the package independently
runnable (uvicorn routes:app). Every route follows the same DataK3 rules the package bakes in,
so quoting it is documenting them.
The connection and the one write helper live in db.py. A DataK3 bucket is a Postgres
endpoint — db name = the bucket, user = the literal token, password = your DODIL token — so
there's no data connect step in code, just fixed region constants. upsert() is the only
writer every route uses:
# db.py — INSERT ... ON CONFLICT DO UPDATE (idempotent keyed write); DO NOTHING for pure edge rows
def upsert(session, model, rows, key):
keys = [key] if isinstance(key, str) else list(key)
table = model.__table__
# normalise to a uniform column set — a multi-row VALUES needs every row to name the
# same columns; fill any a caller omitted with None.
cols = {c for r in rows for c in r}
rows = [{c: r.get(c) for c in cols} for r in rows]
stmt = pg_insert(table).values(rows)
update_cols = [c.name for c in table.columns if c.name not in keys and c.name in cols]
if update_cols:
stmt = stmt.on_conflict_do_update(
index_elements=keys,
set_={c: getattr(stmt.excluded, c) for c in update_cols},
)
else:
stmt = stmt.on_conflict_do_nothing(index_elements=keys)
session.execute(stmt)Why it matters: on DataK3 a bare re-INSERT of an already-committed primary key raises
duplicate-key 23505 — a plain INSERT is not an upsert on re-write. upsert makes a retry,
a shard replay, or a re-import land the row once. That's the whole reason the discovery
warehouse and the convert step are safe to re-run.
Warehouse CRUD — each write is an upsert then a commit (DataK3 has no read-your-writes
inside an open transaction; the engine is expire_on_commit=False, so routes commit before they
return):
# routes.py — CRUD over the discovery warehouse, keyed on the natural PK (domain)
@router.post("/organizations")
def upsert_organization(o: OrganizationIn, s: Session = Depends(db)):
upsert(s, Organization, [o.model_dump()], key="domain")
s.commit()
return {"ok": True, "domain": o.domain}
@router.get("/organizations/{domain}")
def get_organization(domain: str, s: Session = Depends(db)):
row = s.get(Organization, domain)
if not row:
raise HTTPException(404, "no such organization")
return row.__dict__ | {"_sa_instance_state": None}Workflow op 1 — the cost gate (Ignite Models). POST /organizations/qualify classifies
only the bounded batch you pass (the cost ceiling) on kimi-k2.6, then writes the verdict
back onto each organizations row. The write-back reads the row and upserts the full merged
record, so the discovery fields (category, description, …) survive — only
buyer/fit_score/angle/verified change. kimi-k2.6 is a reasoning model, so max_tokens
is high (else content comes back empty) and the response is wrapped in data:
# routes.py — workflow op 1: qualify a BOUNDED batch on Ignite Models, write the verdict back.
# This is one of the suite's four role gates: it spends model money, so it demands the
# orgs:qualify pool permission (see ## Auth) on top of the gateway's login.
@router.post("/organizations/qualify")
def qualify_batch(q: QualifyIn,
user: dict = Depends(require_permission("orgs:qualify")),
s: Session = Depends(db)):
token = _models_token()
verdicts = []
for domain in q.domains:
org = s.get(Organization, domain)
if not org:
continue
v = _classify(token, org, q.product_pitch)
row = {c.name: getattr(org, c.name) for c in Organization.__table__.columns}
row.update(buyer=v["buyer"], fit_score=float(v["fit_score"]), angle=v["angle"], verified=True)
upsert(s, Organization, [row], key="domain") # merged full row -> discovery fields intact
verdicts.append({"domain": domain, "buyer": v["buyer"], "fit_score": float(v["fit_score"])})
s.commit() # commit before any read-back
return {"qualified": len(verdicts), "verdicts": verdicts}Live-verified 2026-09-06 on the persistent crm bucket, over the suite's seeded funnel:
POST /organizations/qualify on a three-org batch returned greyparrot.ai strong 0.95,
corvid.ai strong 0.92, tinyshop.example weak 0.05 — the toy shop correctly rejected —
and the write-back left every discovery field intact. One operational note: each kimi-k2.6 call runs
~12s to nearly 3 minutes, so the loop classifies, then writes — never hold a pg connection open across a
model call.
Workflow op 2 — expand the target list by lookalike (VECTOR). POST /organizations/expand
takes a strong-buyer profile embedding and returns the nearest organizations by pgvector cosine
distance over org_embeddings — the same "find more like these" as Step 3, callable from your app:
# routes.py — workflow op 2: lookalike expansion (VECTOR)
@router.post("/organizations/expand")
def expand_lookalikes(q: ExpandIn, s: Session = Depends(db)):
rows = s.execute(
select(OrgEmbedding.domain, OrgEmbedding.embedding.cosine_distance(q.embedding).label("d"))
.order_by("d")
.limit(q.top_k)
).all()
return {"matches": [{"domain": d, "distance": float(dist)} for d, dist in rows]}Live-verified 2026-09-06 over the indexed org_embeddings: the strong-buyer profile's nearest
lookalike came back at cosine 0.3496, with the strong buyers clustering ahead of the weak org —
the top of that list is your lookalike audience, grown for the price of one embedding.
Workflow op 3 — convert a strong buyer (SQL writes). POST /organizations/{domain}/convert
refuses anything but buyer='strong', then writes a lead + an opportunity tied together by a
conversions audit row. Every write is a keyed upsert, so re-running the conversion is
idempotent — one lead, one opportunity, one conversions row per domain, no matter how many retries:
# routes.py — workflow op 3: convert a strong org into a lead + opportunity (+ conversions audit)
@router.post("/organizations/{domain}/convert")
def convert_organization(domain: str, c: ConvertIn, s: Session = Depends(db)):
org = s.get(Organization, domain)
if not org:
raise HTTPException(404, "no such organization")
if org.buyer != "strong":
raise HTTPException(409, f"organization is buyer={org.buyer!r}, not 'strong' — not convertible")
now = _now()
lead_id, opp_id = f"lead:{domain}", f"opp:{domain}"
upsert(s, Lead, [{ ... }], key="lead_id") # status qualified, score = org.fit_score
upsert(s, Opportunity, [{ ... }], key="opportunity_id")
upsert(s, Conversion, [{ ... }], key="conversion_id") # conv:<domain> -> lead + opp + contact
s.commit()
return {"ok": True, "domain": domain, "lead_id": lead_id, "opportunity_id": opp_id}Live-verified 2026-09-06: converting the strong orgs wrote qualified leads + opportunities +
conversions; re-running the convert re-wrote the same keys (INSERT … ON CONFLICT DO UPDATE), so
the conversion count held. And the tier gate is enforced in the route, not the docs: a convert of the
weak buyer tinyshop.example was refused with 409 (buyer='weak', not 'strong' — not convertible).
Adding a new business operation touches only routes.py (and maybe models.py) — the
plumbing in db.py is fixed. The pattern is one Pydantic *In schema + one @router.<verb>
function: write via upsert, vector via cosine_distance(…), graph via a top-level cypher(…)
fed into a SQL IN (…) (see EXTENDING.md in the package).
Auth — config at the edge, one gate in the app
On Ignite, end-user login is configuration, not code. The suite deploys with the crm-suite
dodil-appid pool attached (user_pool: crm-suite in .dodil/deploy.yaml, issuer
https://appid.dodil.io/ihdiash/crm-suite) and the per-cluster Ignite gateway runs the entire
browser login at the edge — PKCE S256 against the pool issuer, an AEAD-sealed session cookie, JWT
verification — then injects the verified identity into every request: X-Dodil-User (sub, email,
app_roles) and X-Dodil-User-Jwt (the raw verified token, carrying the catalog-expanded
permissions claim). Inbound copies of those headers are stripped, so they can't be forged. The
package's auth.py is therefore a header-trust reader, not a verifier — no JWKS client, no
issuer/audience env, no crypto dependency:
# auth.py — a header-trust reader, not a verifier (the gateway already checked the JWT)
def current_user(request: Request) -> dict: ... # reads x-dodil-user (+ permissions off the JWT)
def require_permission(perm: str): ... # Depends-factory: 403 without the permissionWhat survives in the routes is role-based gating — and after the audit, this component kept exactly
one gate: POST /organizations/qualify demands the orgs:qualify permission, because it's the
route that spends model money (a kimi-k2.6 call per org in the batch). Everything else — warehouse
CRUD, the expand KNN, even the convert (idempotent, and already tier-gated on buyer='strong' in the
data) — rides on the gateway's authentication alone. The suite's other surviving gates live in the
sibling components: leads:score (qualification-scoring), quotes:approve (quote-cpq),
forecast:override (pipeline-forecast). All are checked against the pool's sales / analyst /
manager role catalog — analyst and manager carry orgs:qualify.
The two-plane rule is unchanged: the pool identifies the user; the app still reaches DataK3
through its own service account — an app-user is never a bucket principal. Locally (no gateway in
front of uvicorn routes:app) opt in to a stub identity with DEV_ALLOW_ANON=1; the stub carries
no permissions unless you grant them (DEV_USER_PERMISSIONS=orgs:qualify), so the gate stays gated
even on a laptop. Pool creation, the redirect_uris allowlist, and the off-gateway path where you do
verify the pool JWT yourself (iss and aud mandatory): App
authentication; the catalog mechanics: App
roles.
Get the code
The package is a real download — code/crm-lead-to-opportunity/v1.tar.
This post is a walkthrough of exactly those files; the tarball is source only (no Dockerfile — the
tick/tracking engines and their deploy live in Steps 6–7):
models.py # SQLAlchemy — organizations (+ org_embeddings), the flow tables, and the
# crm/core masters this converts into (stubbed)
routes.py # FastAPI — an APIRouter the suite mounts + a standalone app; warehouse CRUD
# + qualify (Models, orgs:qualify-gated) + expand (vector) + convert
db.py # lazy engine (openapi() builds with no creds) + the ON CONFLICT upsert helper
auth.py # gateway header-trust: current_user + require_permission — no verifier
sa_token.py # deployed: mints + refreshes the service-account token for the pg-wire password
.env.example # BUCKET + PG_HOST/PG_PORT + DODIL_TOKEN (or the SA pair) + DEV_ALLOW_ANON
requirements.txt # sqlalchemy, psycopg[binary], pgvector, fastapi, uvicorn, pydantic, httpx
Run it — point .env at your bucket, create the tables from the models, serve:
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env # then set DODIL_TOKEN (your `dodil auth login` token)
# BUCKET defaults to "crm"; create it once (Step 1), then create the tables from the models:
python -c "import db, models; models.Base.metadata.create_all(db.engine)"
uvicorn routes:app --reload
# POST /organizations · GET /organizations/{domain} · POST /organizations/qualify
# POST /organizations/expand · POST /organizations/{domain}/convertmodels.Base.metadata.create_all is the ORM tab's every class at once — the same tables
Steps 1–5 built by CLI, created from the natural-key models with no migration tool.
In the suite, this component doesn't run alone: the seven crm/* packages compose into one
Ignite app — crm-suite-app mounts each component's APIRouter under a per-component prefix (this
one at /lead-to-opportunity) on one FastAPI process, over one engine to the one crm bucket,
deployed through the git cycle with user_pool: crm-suite — see
Ship a DODIL app. The tick engine and the tracking app below stay
their own deploys — a public recipient-facing surface and the private suite are a real trust-boundary
split.
Step 6 — Deploy the tick engine (Ignite, private)
A small Ignite app walks the sequence. It ships as an image-mode HTTP server (its own Dockerfile;
the platform builds it on deploy — Lane B) and gets its own service account to reach DataK3. That SA's
client_credentials access token is both the bearer for id.dodil.io and the Postgres-wire password —
there is no K3 HTTP API; the data plane is the drop-in pg wire at pg.uk-lon-1.dodil.io:5432 (dbname=<bucket>,
user=token), driven with psycopg. Each tick (a POST /tick) grabs the enrollments that are due, sends
the current flow_step (gated on contacts.subscribed, so an unsubscribe is honored instantly), logs
flow_actions + activities, and advances current_position / next_run_at. Ignite is request-invoked
(scale-to-zero) with no server-side scheduler — drive /tick from your own cron, or pin an always-on poll
loop warm with --auto-min-instances 1.
# seq-engine/server.py — IMAGE-mode Ignite app (HTTP server on $PORT), PRIVATE. Calls NO Models.
# GET /healthz -> 200 {"status":"ready"} (probe path; no auth, no DataK3)
# POST /tick -> body {"limit": N} (walks one tick of the nurture flow)
# All reads/writes go over the DROP-IN POSTGRES WIRE — the SA access token is the pg password.
from datetime import datetime, timezone
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from psycopg import errors as pg_errors
BUCKET = os.environ["BUCKET"]
SA_ID = os.environ["DODIL_SERVICE_ACCOUNT_ID"] # the cli-… serviceAccountId, NOT the uuid
SA_SECRET = os.environ["DODIL_SERVICE_ACCOUNT_SECRET"]
PG_HOST = os.environ.get("PG_HOST", "pg.uk-lon-1.dodil.io")
PG_PORT = int(os.environ.get("PG_PORT", "5432"))
ID_URL = "https://id.dodil.io/realms/dodil/protocol/openid-connect/token"
# an explicit User-Agent is REQUIRED — stdlib urllib's default "Python-urllib/x" is
# banned by Cloudflare at id.dodil.io (HTTP 403 "error code: 1010").
UA = "crm-sequence-engine/1.0"
def _now():
return datetime.now(timezone.utc).isoformat()
def _http_post(url, data, headers, form=False):
headers = {"User-Agent": UA, **headers}
body = urllib.parse.urlencode(data).encode() if form else json.dumps(data).encode()
headers["Content-Type"] = "application/x-www-form-urlencoded" if form else "application/json"
req = urllib.request.Request(url, data=body, headers=headers, method="POST")
with urllib.request.urlopen(req, timeout=90) as r:
return json.loads(r.read().decode())
def _token():
out = _http_post(ID_URL, {"grant_type": "client_credentials",
"client_id": SA_ID, "client_secret": SA_SECRET}, headers={}, form=True)
return out["access_token"]
def _pg(token):
# drop-in Postgres wire: DB name = bucket, user "token", password = the SA access token.
return psycopg.connect(host=PG_HOST, port=PG_PORT, dbname=BUCKET, user="token",
password=token, sslmode="require", connect_timeout=20, autocommit=False)
def _retry(fn):
# the pg engine is serializable — retry a write on a transient serialization/deadlock.
for attempt in range(4):
try:
return fn()
except (pg_errors.SerializationFailure, pg_errors.DeadlockDetected):
if attempt == 3:
raise
time.sleep(0.4 * (attempt + 1))
def _subscribed(token, email):
with _pg(token) as conn, conn.cursor() as cur:
cur.execute("SELECT subscribed FROM contacts WHERE email = %s", (email,))
row = cur.fetchone()
return bool(row and row[0])
def _next_run(delay):
return datetime.fromtimestamp(time.time() + int(delay or 0), timezone.utc).isoformat()
def _send_email(to, subject, body_html):
# hand off to YOUR ESP (SES / SendGrid / Postmark …) — the one intentional stub.
# Everything around it (the DataK3 reads/writes) is real.
pass
def _tick(token, limit):
now = _now()
with _pg(token) as conn, conn.cursor() as cur:
cur.execute("SELECT enrollment_id, flow_id, contact_email, opportunity_id, current_position "
"FROM flow_enrollments WHERE status = 'active' AND next_run_at <= %s "
"ORDER BY next_run_at LIMIT %s", (now, limit))
due = cur.fetchall()
sent = skipped = done = 0
for enr_id, flow_id, email, opp_id, pos in due:
with _pg(token) as conn, conn.cursor() as cur:
cur.execute("SELECT step_id, kind, subject, body_html, delay_seconds "
"FROM flow_steps WHERE flow_id = %s AND position = %s", (flow_id, pos))
step = cur.fetchone()
if step is None: # walked off the end -> done
def _finish():
with _pg(token) as c, c.cursor() as cur:
cur.execute("UPDATE flow_enrollments SET status = 'done', next_run_at = %s "
"WHERE enrollment_id = %s", (now, enr_id))
c.commit()
_retry(_finish); done += 1
continue
step_id, kind, subject, body_html, delay = step
gated = kind == "email" and not _subscribed(token, email) # honor unsubscribe
status = "skipped:unsubscribed" if gated else "sent"
# action_id / activity_id are deterministic per (enrollment, position) — a poll retry or shard
# re-entry re-writes the SAME PK, so these must be ON CONFLICT upserts to be re-run safe. A bare
# re-INSERT of a committed PK raises duplicate-key 23505; DuckDB pg-wire supports ON CONFLICT.
def _log_action():
with _pg(token) as c, c.cursor() as cur:
cur.execute("INSERT INTO flow_actions "
"(action_id, enrollment_id, flow_id, step_id, contact_email, kind, status, detail, ts) "
"VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s) "
"ON CONFLICT (action_id) DO UPDATE SET enrollment_id = EXCLUDED.enrollment_id, "
"flow_id = EXCLUDED.flow_id, step_id = EXCLUDED.step_id, "
"contact_email = EXCLUDED.contact_email, kind = EXCLUDED.kind, "
"status = EXCLUDED.status, detail = EXCLUDED.detail, ts = EXCLUDED.ts",
(f"act:{enr_id}:{pos}", enr_id, flow_id, step_id, email, kind, status, subject, now))
c.commit()
_retry(_log_action)
if not gated:
def _log_activity():
with _pg(token) as c, c.cursor() as cur:
cur.execute("INSERT INTO activities "
"(activity_id, opportunity_id, contact_email, kind, subject, body, direction, status, ts) "
"VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s) "
"ON CONFLICT (activity_id) DO UPDATE SET opportunity_id = EXCLUDED.opportunity_id, "
"contact_email = EXCLUDED.contact_email, kind = EXCLUDED.kind, "
"subject = EXCLUDED.subject, body = EXCLUDED.body, "
"direction = EXCLUDED.direction, status = EXCLUDED.status, ts = EXCLUDED.ts",
(f"eml:{enr_id}:{pos}", opp_id, email, "email", subject, body_html,
"outbound", "sent", now))
c.commit()
_retry(_log_activity)
_send_email(email, subject, body_html)
sent += 1
else:
skipped += 1
# advance the existing enrollment row (UPDATE — like moving leads.status in the scorer)
def _advance():
with _pg(token) as c, c.cursor() as cur:
cur.execute("UPDATE flow_enrollments SET current_position = %s, next_run_at = %s "
"WHERE enrollment_id = %s", (pos + 1, _next_run(delay), enr_id))
c.commit()
_retry(_advance)
return {"ticked": len(due), "sent": sent, "skipped": skipped, "done": done}
class Handler(BaseHTTPRequestHandler):
def _send(self, code, body):
payload = json.dumps(body).encode()
self.send_response(code)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(payload)))
self.end_headers()
self.wfile.write(payload)
def do_GET(self):
if self.path == "/healthz":
return self._send(200, {"status": "ready"})
return self._send(404, {"error": "no_route", "path": self.path})
def do_POST(self):
if self.path != "/tick":
return self._send(404, {"error": "no_route", "path": self.path})
try:
n = int(self.headers.get("Content-Length") or 0)
body = json.loads(self.rfile.read(n) or b"{}")
return self._send(200, _tick(_token(), int(body.get("limit", 200))))
except urllib.error.HTTPError as e:
return self._send(502, {"error": "upstream", "code": e.code,
"body": e.read().decode(errors="replace")[:600]})
except Exception as e:
return self._send(500, {"error": type(e).__name__, "detail": str(e)[:600]})
def log_message(self, *a):
pass
if __name__ == "__main__":
port = int(os.environ.get("PORT", "8080"))
print(f"crm-sequence-engine serving on 0.0.0.0:{port} bucket={BUCKET}", flush=True)
ThreadingHTTPServer(("0.0.0.0", port), Handler).serve_forever()The image is tiny — the pg driver is the only third-party dep:
# seq-engine/Dockerfile
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY server.py .
ENV PORT=8080
EXPOSE 8080
CMD ["python", "server.py"]# seq-engine/requirements.txt
psycopg[binary]==3.2.3Give the engine its own least-privilege identity, then deploy it in image mode (--dockerfile-path, no
--runtime). It writes tables but calls no Models, so it needs k3.editor (write DataK3) +
ignite.app-developer (the deploy identity) — not ignite.model-user.
Create a service account crm-sequence-engine-sa, grant it k3.editor (write tables) and ignite.app-developer (deploy identity), then deploy my ./seq-engine app to Ignite as crm-sequence-engine in IMAGE mode (its Dockerfile, port 8080, health /healthz, private) with the SA creds + BUCKET as runtime env, and POST one tick as a smoke test.
auth_service-account_create→auth_service-account_grant-role→ignite_app_deploy→ignite_app_getCreated crm-sequence-engine-sa (use its cli- serviceAccountId, not the uuid); granted k3-authorization-service k3.editor + ignite-authorization-service ignite.app-developer. Built and deployed crm-sequence-engine from its Dockerfile (image mode, private, scale-to-zero). POST /tick returned {"ticked":1,"sent":1,"skipped":0,"done":0} — it sent the due step and advanced the enrollment.
dodil auth service-account create crm-sequence-engine-sa
# Use the serviceAccountId it prints — a cli-… id, NOT the uuid. The uuid fails
# client_credentials with invalid_client; the cli- id is the working client_id.
SA_ID=$(dodil auth service-account list -o json | python3 -c "import sys,json;print([s['serviceAccountId'] for s in json.load(sys.stdin) if 'crm-sequence-engine-sa' in s['serviceAccountId']][0])")
# NO ignite.model-user — this engine writes tables over pg-wire but calls no Models
dodil auth service-account grant-role "$SA_ID" k3-authorization-service k3.editor
dodil auth service-account grant-role "$SA_ID" ignite-authorization-service ignite.app-developer
# IMAGE mode: the platform builds ./seq-engine/Dockerfile on deploy (Lane B) and runs the HTTP server.
# Private (no --allow-unauthenticated) — the tick route is driven by your own authenticated cron.
dodil ignite app deploy crm-sequence-engine \
--code ./seq-engine --dockerfile-path Dockerfile \
--port 8080 --health-path /healthz \
--env DODIL_SERVICE_ACCOUNT_ID=$SA_ID --env DODIL_SERVICE_ACCOUNT_SECRET=$SA_SECRET \
--env BUCKET="$BUCKET"
# drive a tick from your scheduler (private app -> pass a bearer token; or pin warm with --auto-min-instances 1)
BASE=$(dodil ignite app get crm-sequence-engine --output json | python3 -c "import sys,json;print(json.load(sys.stdin)['public_urls'][0])")
curl -s -X POST "https://$BASE/tick" -H "Authorization: Bearer $DODIL_TOKEN" \
-d '{"limit":50}' # {"ticked":1,"sent":1,"skipped":0,"done":0}Step 7 — Track opens, clicks & unsubscribes (public/private split)
Recipients hit three routes — an open pixel, a click redirect, and unsubscribe — and none should
sit next to your admin secrets. So they run as a separate, minimal Ignite app: auth-less, the fewest secrets
possible, writing an email_events row and honoring opt-outs by flipping contacts.subscribed. Its only
secret beyond DataK3 access is the unsubscribe signing key.
In crm, create an email_events table keyed on event_id with action_id, enrollment_id, contact_email, type, detail, ts.
data_table_createCreated email_events (key event_id) — the public tracking app writes an 'open' or 'click' row per hit; unsubscribe flips contacts.subscribed and logs an activity.
dodil data table create email_events -b "$BUCKET" --merge-key event_id \
--columns-json '[{"name":"event_id","type":"string","nullable":false},{"name":"action_id","type":"string"},{"name":"enrollment_id","type":"string"},{"name":"contact_email","type":"string"},{"name":"type","type":"string"},{"name":"detail","type":"string"},{"name":"ts","type":"string"}]'# tracking/server.py — IMAGE-mode Ignite app (HTTP server on $PORT), PUBLIC (no auth). Calls NO Models.
# The open/click/unsubscribe links real recipients hit — kept OFF the admin box, carrying the fewest
# secrets possible: DataK3 pg-wire access + the unsubscribe signing key (the ONLY secret beyond DataK3).
# GET /healthz -> 200 {"status":"ready"} (probe; no token, no DataK3)
# GET /track/open?a=&c= -> writes an 'open' email_events row, returns a 1x1 gif
# GET /track/click?a=&c=&u= -> writes a 'click' row, 302-redirects to u
# GET /unsubscribe?e=&t= -> verifies the HMAC, flips contacts.subscribed, logs it
from datetime import datetime, timezone
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from psycopg import errors as pg_errors
BUCKET = os.environ["BUCKET"]
SA_ID = os.environ["DODIL_SERVICE_ACCOUNT_ID"] # the cli-… serviceAccountId, NOT the uuid
SA_SECRET = os.environ["DODIL_SERVICE_ACCOUNT_SECRET"]
UNSUB_KEY = os.environ["UNSUB_SIGNING_KEY"].encode() # the ONLY secret beyond DataK3 access
PG_HOST = os.environ.get("PG_HOST", "pg.uk-lon-1.dodil.io")
PG_PORT = int(os.environ.get("PG_PORT", "5432"))
ID_URL = "https://id.dodil.io/realms/dodil/protocol/openid-connect/token"
UA = "crm-tracking/1.0" # explicit UA — urllib's default is Cloudflare-banned at id.dodil.io (403)
GIF = (b"GIF89a\x01\x00\x01\x00\x80\x00\x00\xff\xff\xff\x00\x00\x00!\xf9\x04\x01\x00\x00"
b"\x00\x00,\x00\x00\x00\x00\x01\x00\x01\x00\x00\x02\x02D\x01\x00;") # 1x1 transparent pixel
def _now():
return datetime.now(timezone.utc).isoformat()
def _http_post(url, data, headers, form=False):
headers = {"User-Agent": UA, **headers}
body = urllib.parse.urlencode(data).encode() if form else json.dumps(data).encode()
headers["Content-Type"] = "application/x-www-form-urlencoded" if form else "application/json"
req = urllib.request.Request(url, data=body, headers=headers, method="POST")
with urllib.request.urlopen(req, timeout=90) as r:
return json.loads(r.read().decode())
def _token():
out = _http_post(ID_URL, {"grant_type": "client_credentials",
"client_id": SA_ID, "client_secret": SA_SECRET}, headers={}, form=True)
return out["access_token"]
def _pg(token):
# drop-in Postgres wire: DB name = bucket, user "token", password = the SA access token.
return psycopg.connect(host=PG_HOST, port=PG_PORT, dbname=BUCKET, user="token",
password=token, sslmode="require", connect_timeout=20, autocommit=False)
def _retry(fn):
for attempt in range(4):
try:
return fn()
except (pg_errors.SerializationFailure, pg_errors.DeadlockDetected):
if attempt == 3:
raise
time.sleep(0.4 * (attempt + 1))
def _sign(email):
return hmac.new(UNSUB_KEY, email.encode(), hashlib.sha256).hexdigest()
def _event(ev_type, action_id, email, detail):
now = _now()
def _w():
with _pg(_token()) as c, c.cursor() as cur:
# event_id carries the timestamp -> a fresh PK every event (append-only), so this first-write
# INSERT is correct. (Re-writing the SAME key would instead need ON CONFLICT DO UPDATE — a bare
# re-INSERT of a committed PK raises duplicate-key 23505.)
cur.execute("INSERT INTO email_events (event_id, action_id, contact_email, type, detail, ts) "
"VALUES (%s,%s,%s,%s,%s,%s)",
(f"ev:{ev_type}:{action_id or email}:{now}", action_id, email, ev_type, detail, now))
c.commit()
_retry(_w)
def _unsubscribe(email):
now = _now()
def _w():
with _pg(_token()) as c, c.cursor() as cur:
cur.execute("UPDATE contacts SET subscribed = false, updated_at = %s WHERE email = %s",
(now, email))
cur.execute("INSERT INTO activities (activity_id, contact_email, kind, direction, status, ts) "
"VALUES (%s,%s,%s,%s,%s,%s)",
(f"unsub:{email}:{now}", email, "unsubscribe", "inbound", "done", now))
c.commit()
_retry(_w)
class Handler(BaseHTTPRequestHandler):
def _json(self, code, body):
payload = json.dumps(body).encode()
self.send_response(code)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(payload)))
self.end_headers()
self.wfile.write(payload)
def _raw(self, code, ctype, payload, extra=None):
self.send_response(code)
self.send_header("Content-Type", ctype)
self.send_header("Content-Length", str(len(payload)))
for k, v in (extra or {}).items():
self.send_header(k, v)
self.end_headers()
self.wfile.write(payload)
def do_GET(self):
u = urllib.parse.urlparse(self.path)
q = urllib.parse.parse_qs(u.query)
one = lambda k: (q.get(k) or [""])[0]
try:
if u.path == "/healthz":
return self._json(200, {"status": "ready"}) # no token, no DataK3
if u.path == "/track/open":
_event("open", one("a"), one("c"), None)
return self._raw(200, "image/gif", GIF) # the tracking pixel
if u.path == "/track/click":
_event("click", one("a"), one("c"), one("u"))
return self._raw(302, "text/plain", b"", {"Location": one("u") or "/"})
if u.path == "/unsubscribe":
email, tok = one("e"), one("t")
if not hmac.compare_digest(tok, _sign(email)): # the ONLY secret this app carries
return self._json(403, {"status": "error", "reason": "bad_signature"})
_unsubscribe(email)
return self._raw(200, "text/html; charset=utf-8", b"<h1>You are unsubscribed.</h1>")
return self._json(404, {"error": "no_route", "path": u.path})
except urllib.error.HTTPError as e:
return self._json(502, {"error": "upstream", "code": e.code})
except Exception as e:
return self._json(500, {"error": type(e).__name__, "detail": str(e)[:600]})
def log_message(self, *a):
pass
if __name__ == "__main__":
port = int(os.environ.get("PORT", "8080"))
print(f"crm-tracking serving on 0.0.0.0:{port} bucket={BUCKET}", flush=True)
ThreadingHTTPServer(("0.0.0.0", port), Handler).serve_forever()Same Dockerfile + requirements.txt as the tick engine (only server.py differs):
# tracking/Dockerfile
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY server.py .
ENV PORT=8080
EXPOSE 8080
CMD ["python", "server.py"]# tracking/requirements.txt
psycopg[binary]==3.2.3Create a service account crm-tracking-sa, grant it k3.editor only, then deploy my ./tracking app to Ignite as crm-tracking in IMAGE mode (its Dockerfile, port 8080), public (allow-unauthenticated), with the SA creds + BUCKET + the unsubscribe signing key as runtime env and /healthz as the health path. Then curl its health path with no token.
auth_service-account_create→auth_service-account_grant-role→ignite_app_deploy→ignite_app_getCreated crm-tracking-sa (use its cli- serviceAccountId, not the uuid); granted k3-authorization-service k3.editor only. Built and deployed crm-tracking from its Dockerfile (image mode, public — build-on-deploy works with --allow-unauthenticated, no pull secret) — open pixel, click redirect, unsubscribe. It carries DataK3 pg-wire access + the unsubscribe signing key; every other secret stays in crm-sequence-engine. curl /healthz returns 200 with no token.
dodil auth service-account create crm-tracking-sa
# Use the serviceAccountId it prints — a cli-… id, NOT the uuid (the uuid fails client_credentials).
TSA_ID=$(dodil auth service-account list -o json | python3 -c "import sys,json;print([s['serviceAccountId'] for s in json.load(sys.stdin) if 'crm-tracking-sa' in s['serviceAccountId']][0])")
dodil auth service-account grant-role "$TSA_ID" k3-authorization-service k3.editor # k3.editor ONLY
# IMAGE mode + public: build-on-deploy (Lane B) serves --allow-unauthenticated with no pull secret.
dodil ignite app deploy crm-tracking \
--code ./tracking --dockerfile-path Dockerfile \
--port 8080 --health-path /healthz --allow-unauthenticated \
--env DODIL_SERVICE_ACCOUNT_ID=$TSA_ID --env DODIL_SERVICE_ACCOUNT_SECRET=$TSA_SECRET \
--env BUCKET="$BUCKET" --env UNSUB_SIGNING_KEY=$UNSUB_SIGNING_KEY
BASE=$(dodil ignite app get crm-tracking --output json | python3 -c "import sys,json;print(json.load(sys.stdin)['public_urls'][0])")
curl -s "https://$BASE/healthz" # {"status":"ready"} — 200, no tokenQuery it — the funnel, one bucket, drop-in clients
Now it's a pipeline you query — by content (SQL), by meaning (vector), by relationship (graph) — over one
bucket. And any Postgres/pgvector or Bolt/Neo4j client points straight at it: data connect prints the
endpoints (DB name = bucket, credential = your login token).
In crm, show the discovery funnel (organizations by buyer tier), the converted pipeline (leads by status + count of conversions), and print the endpoints to point psql at the crm bucket.
data_sql→data_connectDiscovery funnel: strong 3, weak 1. Converted: 3 qualified leads, 1 conversion (harborlytics.io fully worked). data connect printed pg pg.uk-lon-1.dodil.io:5432/crm — same bucket, drop-in clients.
# discovery funnel by buyer tier
dodil data sql -b "$BUCKET" \
"SELECT buyer, count(*) AS n FROM organizations GROUP BY buyer ORDER BY n DESC"
# strong 3 | weak 1
# converted pipeline
dodil data sql -b "$BUCKET" \
"SELECT (SELECT count(*) FROM leads WHERE status='qualified') AS qualified_leads,
(SELECT count(*) FROM conversions) AS conversions"
# qualified_leads 3 | conversions 1
# drop-in clients: same bucket, your own psql / cypher-shell / pgvector driver
dodil data connect "$BUCKET" -o psql
# postgresql://token:[email protected]:5432/crm?sslmode=requireHow the pillars map
One bucket, three pillars, one copy of the rows — this front door would otherwise be four systems and the glue between them.
| Job | The usual stack | On DataK3 |
|---|---|---|
| Discovery warehouse + converted leads/opps | Postgres (or a SaaS CRM seat) | SQL tables in the bucket |
| Lookalike audience expansion | Pinecone + an embedding pipeline | org_embeddings VECTOR(2048) + data vsearch |
| Converted org joins the account family | Neo4j + a sync job | works_at edge in crm_graph — graph_khop |
| Qualify only what you'll pay for | A scoring service + a queue | one kimi-k2.6 call per batched org |
| Point your own tools at it | Per-system drivers & creds | data connect — psql / bolt / pgvector, DB = bucket |
No ETL, no second copy, no drift: a discovered org, its embedding, its converted lead, and its graph edge all
join on domain for free, because it's all one bucket.
Customize — the decisions this skill asks you
Q1 · product_pitch — what counts as a fit?
"What are you selling, in one line?" (default: "a unified data backend (SQL+vector+graph in one bucket)")
Rewrites the qualify gate's system prompt (Step 2). This is the money knob — it decides which companies
come back strong and therefore what your entire downstream funnel costs. Change the pitch, change the buyer.
Q2 · qualify_batch — how many per run?
"How many organizations should one gate run classify?" (default: 25)
A hard cap on orgs sent to kimi-k2.6 per run — the cost ceiling. Discovery fills the warehouse for free;
this bounds what you pay to judge. Raise it to burn down a backlog faster; lower it to trickle spend.
Q3 · auto_convert_tier — how wide is the convert gate?
"Convert only
strongbuyers, orstrongandpossible?" (default: strong)
Sets the WHERE buyer = … clause of the convert step (Step 4) and the ## Test count. strong keeps the
funnel tight and cheap; strong_possible widens the top of the pipeline at the cost of more low-fit leads.
Q4 · intake — where do candidates come from?
"Discover into a warehouse, or hand-feed leads?" (default: discovery)
- discovery → Steps 1–3 build the
organizationswarehouse +org_embeddings, and the gate qualifies. - manual → skip the warehouse (no
organizations/org_embeddings); hand-upsertleadsdirectly and start at Step 4. Use when leads arrive from a form or an SDR, already identified.
Q5 · nurture_flow — nurture, or just convert?
"Deploy the email sequence engine, or stop at conversion?" (default: true)
- true → Steps 5–7 build the flow tables + deploy
crm-sequence-engine(private) andcrm-tracking(public). - false → the module is discover → convert only; skip both Ignite apps and the flow/email tables.
Test
Every command below ran live against DataK3 (org IHDIASH) with the values shown — the standalone demo on
2026-09-01, and the package routes re-validated 2026-09-06 on the persistent crm bucket over the
suite's seeded funnel (qualify: greyparrot.ai 0.95 / corvid.ai 0.92 / tinyshop.example 0.05; expand
0.3496; weak-buyer convert → 409). The two Ignite deploys in Steps 6–7 use the image-mode pattern
validated live 2026-09-02 on the companion lead scorer (deploys, serves /healthz unauthenticated,
writes durably — see the note).
# 1. warehouse + flow tables exist (15 incl. crm_node/crm_edge)
dodil data table list -b "$BUCKET"
# organizations, org_embeddings, flows, flow_steps, flow_enrollments, flow_actions, email_events,
# conversions, leads, contacts, accounts, opportunities, activities, crm_node, crm_edge
# 2. the qualify gate returns a valid buyer-tier JSON verdict
dodil ignite models chat kimi-k2.6 \
--system 'Return ONLY JSON: buyer (strong|possible|weak), fit_score (0-1), angle (<=12 words).' \
--message 'org: Harborlytics | harborlytics.io | CDP stitching Postgres, Pinecone, and a warehouse.'
# {"buyer":"strong","fit_score":0.92,"angle":"Replace stitched Postgres/Pinecone/warehouse with unified backend"}
# 3. strong orgs converted to leads + a conversions audit row
dodil data sql -b "$BUCKET" "SELECT
(SELECT count(*) FROM organizations WHERE buyer='strong') AS strong_orgs,
(SELECT count(*) FROM leads WHERE status='qualified') AS qualified_leads,
(SELECT count(*) FROM conversions) AS conversions"
# strong_orgs 3 | qualified_leads 3 | conversions 1
# 4. the enrollment is staged at position 1, due now
dodil data sql -b "$BUCKET" \
"SELECT current_position, status FROM flow_enrollments WHERE contact_email='[email protected]'"
# current_position 1 | status active
# 5. the converted org is in the account graph (works_at reaches the contact)
dodil data pg -b "$BUCKET" "SELECT c.name FROM graph_khop('crm_graph',1,1,'in') g
JOIN crm_node c ON c.id=g.node AND c.kind='contact'"
# Harborlytics Data Team
# 6. lookalike KNN ranks the data-infra buyers ahead of the weak DTC store
dodil data vsearch -b "$BUCKET" -t org_embeddings --column embedding \
--text "fragmented data stack stitching SQL, a vector store, and a graph database" \
--model jina-embeddings-v4 --metric cosine --top-k 4
# harborlytics.io 0.33 | nimbusgraph.dev 0.39 | greyparrot.ai 0.47 | brightmugs.com 0.56Live-validated (tested_branch {intake: discovery, auto_convert_tier: strong, nurture_flow: true}):
assertions 1–6 above — 15 tables, gate JSON, strong→lead+conversion, current_position=1, the works_at
graph traversal, and the lookalike KNN. The {intake: manual, nurture_flow: false} thin branch is the same
convert path with the warehouse and Ignite apps skipped.
NOTE
Deploy pattern: image mode, Lane B (build-on-deploy) — validated live 2026-09-02. Steps 6–7 ship both
engines as image-mode HTTP apps (a Dockerfile + --dockerfile-path, not --runtime python).
Validated live 2026-09-02: this pattern deploys, serves /healthz + its route unauthenticated (public
FQDN, /healthz 200, no pull secret with --allow-unauthenticated), and writes durably — confirmed
end-to-end via the companion lead scorer (written rows survived
+154s, re-confirmed at +95s). Both engines here reuse that identical server/helper/deploy skeleton over
pg-wire. The idempotent re-convert and data connect endpoint proof are keyed upserts / a read, safe
to re-run.
If dodil ignite app deploy returns IAM resource registration failed: broken pipe, deploy each engine
under a fresh app name — the half-created app can't be updated or deleted (UMA can't authorize an
unregistered resource).
One-shot
With the DODIL MCP connected, paste this to build the whole front door at once:
Build the discover → qualify → nurture → convert front door of a CRM on DataK3 (one bucket = SQL + vector +
graph). Confirm each step. Assume crm/core masters exist (or stub leads/contacts/accounts/opportunities/
activities first).
1. Create a DataK3 bucket `crm`. Discovery warehouse: organizations (key domain) + org_embeddings
(key domain, VECTOR(2048)). Upsert ~4 discovered companies, buyer=unqualified.
2. Qualify a bounded batch (25) on kimi-k2.6 — system prompt: "Qualify a company as a sales lead for a
unified data backend (SQL+vector+graph in one bucket). Return ONLY JSON: buyer (strong|possible|weak),
fit_score (0-1), angle (<=12 words)." Merge buyer/fit_score/angle back onto each organizations row.
3. Embed each description with jina-embeddings-v4 into org_embeddings (one vector per upsert). Vsearch a
strong-buyer profile for lookalikes.
4. Convert buyer='strong' orgs into leads (status qualified). Fully convert the top one: account + contact +
opportunity + a conversions row. Project account/contact nodes + a works_at edge into crm_node/crm_edge,
then CREATE GRAPH crm_graph; traverse graph_khop('crm_graph',<acct>,1,'in') to confirm.
5. Sequence tables: flows, flow_steps, flow_enrollments, flow_actions. Author an `onboarding` flow (3 email
steps, delays 0 / 3d / 7d) and enroll the converted contact at position 1, due now.
6. Deploy Ignite `crm-sequence-engine` (own SA: k3.editor + ignite.app-developer): a tick loop over due
enrollments that sends the current step (skip if contacts.subscribed is false) and advances position.
7. Create email_events; deploy a SECOND public Ignite app `crm-tracking` (own SA: k3.editor only + the
unsubscribe signing key) serving /healthz, /track/open, /track/click, /unsubscribe.Ship it — crm-tracking as a public endpoint
The tracking app is the public face — the open/click/unsubscribe links real recipients hit. Ship it the last
mile with the same image-mode deploy: the platform builds ./tracking/Dockerfile on deploy (Lane B) and
returns a public FQDN, callable with no token. Build-on-deploy serves --allow-unauthenticated with no pull
secret, and its serviceAccountId (the cli-… id, never the uuid) is what the runtime uses as its DataK3
pg-wire credential.
Deploy crm-tracking publicly with no auth, in image mode from its Dockerfile, and give me its URL and health check.
ignite_app_deploy→ignite_app_getBuilt and deployed crm-tracking from its Dockerfile (image mode); public FQDN on ignite.dodil.cloud, /healthz returns 200 with no token — open/click/unsubscribe links resolve for real recipients.
# IMAGE mode (Dockerfile + --dockerfile-path, no --runtime); public build-on-deploy needs no pull secret
dodil ignite app deploy crm-tracking \
--code ./tracking --dockerfile-path Dockerfile \
--port 8080 --health-path /healthz --allow-unauthenticated \
--env DODIL_SERVICE_ACCOUNT_ID=$TSA_ID --env DODIL_SERVICE_ACCOUNT_SECRET=$TSA_SECRET \
--env BUCKET="$BUCKET" --env UNSUB_SIGNING_KEY=$UNSUB_SIGNING_KEY
dodil ignite app get crm-tracking --output json # -> public_urlsThe full lifecycle — DODIL git → CI checks → a scanned image in the registry → versioning and rollback — is walked end to end in Ship a DODIL App.
Connect your tools
Everything this build wrote lives in one DataK3 bucket, reachable by your own stack — not just the CLI. data connect crm prints the endpoints; point your tools straight at the same rows:
- SQL over Postgres wire —
psql,sqlx/diesel(Rust),psycopg/asyncpg(Python),node-postgres(TS). - Graph over Bolt — a Neo4j driver or
cypher-shellagainstcrm_graph. - Vector — pgvector (
<=>) over the same wire, or a Qdrant/Pinecone client against the sameorg_embeddingsrows.
Full, live-validated walkthrough: Connect your tools.
Conclusion
The front door of a CRM — discover → qualify → nurture → convert — as one DataK3 bucket + two Ignite apps. A discovery warehouse you fill for free, a cost gate that classifies only the batch you choose, a lookalike audience you grow with one embedding, strong buyers converted idempotently into leads and opportunities (and into the account graph), and an email sequence that is rows you edit — walked by a scale-to-zero engine, tracked by a minimal public app. One bucket, one bill, three pillars over one copy of the rows.
Next steps:
- Build a CRM on DataK3 — the
crm/coremasters + the account-hierarchy graph. - Leads Data Warehouse — the discovery + classification engine, in depth.