What you'll build: a shared code knowledge base for a whole engineering org. A GitHub webhook streams every push and PR — across all repos and branches — into one DataK3 bucket, where the code embeds for intent search, symbol metadata lands in SQL, and design decisions (ADRs/PRs) sit next to the code they explain. Every developer's agent queries the same, always-current index over the DODIL MCP.

Why a shared index?

A real org's code sprawls across dozens of repos and hundreds of active branches. The tools each have a gap:

  • git grep and GitHub search are lexical — they match tokens, not intent ("where do we debounce reconnects?").
  • IDE / Copilot indexes are per-developer and per-branch — everyone re-indexes the same code, and it's stale the moment a teammate pushes.
  • The "why" — the ADR, the PR discussion — lives detached from the code it governs.

For teams juggling many repositories, cross-repo and cross-branch intelligence is the deciding factor. One shared, intent-searchable, always-current index — synced from GitHub automatically and queried by every agent — is the payoff. This is that, in one bucket.

What you'll learn:

  • Sync GitHub → DataK3 continuously with a webhook + an Ignite handler authed by a service account.
  • Keep a branch-aware symbol table so search scopes to one branch or spans them all.
  • Answer "where do we do X, on any branch, and why" across objects + Vector + SQL, CLI or agent.

NOTE

Connect the DODIL MCP once — see the two-minute setup. Each step below shows the CLI and an Ask your agent tab.

Problem

A developer — or the coding agent working on their behalf — burns real time answering "where do we do X, on which branch, and why?" git grep is lexical and single-branch, IDE indexes are per-developer and stale the moment a teammate pushes, and the ADR that explains the decision lives in a wiki detached from the code. So the same questions get re-asked, the same code gets re-indexed, and the reasoning is lost. This tutorial gives that developer's agent one shared, always-current index — code embedded for intent search, symbols in SQL scoped by branch, and the "why" (ADRs + PR descriptions) sitting next to the code — so any agent on the team answers "where + why, on any branch" in one query instead of a manual hunt across repos and tools.

Prerequisites

  • A DODIL organization, and the dodil CLI authenticated (dodil auth login) or the DODIL MCP connected to your agent.
  • A GitHub org/repo where you can add a webhook (admin on the repo).

Step 1 — Create the shared bucket + recipe collections

One bucket for the whole org. Two recipes: code-rag for source (AST-aware chunking) and document-rag for ADRs and PR descriptions. Each recipe stands up a collection — a table with a jina-embeddings-v4 vector column — plus the ingest rule that auto-embeds anything landing under its folder. There's no separate vector store to provision.

You

Create a DataK3 bucket called devbrain, then install code-rag with prefix dev scoped to repos/ and document-rag with prefix notes scoped to notes/.

DODIL MCP tools called
data_bucket_create→data_recipe_install
Agent

Created devbrain. Installed code-rag (collection dev_code, AST-aware, jina-embeddings-v4) scoped to repos/, and document-rag (collection notes_docs) scoped to notes/. Anything synced under repos/ embeds into dev_code; ADRs under notes/ embed into notes_docs.

IMPORTANT

One collection for all branches — not one per branch. The code-rag rule is scoped to the repos/ folder, so every object — regardless of branch — embeds into the single dev_code collection. You scope by branch with the symbols table's branch column (Step 3), not by creating a collection per branch. A collection-per-branch would mean hundreds of short-lived collections to spin up and tear down, re-embedding files that are identical across branches, and fanning every cross-branch query across all of them. One collection + branch as metadata lets you search a single branch or span them all. And because objects are keyed by commit (Step 4), a file that's identical across branches is a single object — embedded once, not once per branch. (Reserve a separate collection for a real isolation boundary — say, per repo when languages/embedding models differ — never per branch.)

Step 2 — A service account for the sync (CI identity)

The sync runs in CI and in a webhook handler — no human at a browser — so it authenticates with a service account, not your login. Create one, grant it the least-privilege DataK3 write role, and store the ID + secret where the handler can read them as DODIL_SERVICE_ACCOUNT_ID / DODIL_SERVICE_ACCOUNT_SECRET.

You

Create a service account called ci-code-sync and grant it k3.editor so my GitHub sync can write to devbrain non-interactively.

DODIL MCP tools called
auth_service-account_create→auth_service-account_grant-role
Agent

Created service account ci-code-sync (secret shown once — store it now). Granted k3-authorization-service/k3.editor — enough to write objects and tables, nothing more. The webhook handler and CI can now auth headlessly via the DODIL_SERVICE_ACCOUNT_ID/_SECRET env vars.

TIP

Scope it down and rotate it. Grant only what the sync needs (k3.editor, not k3.admin), and dodil auth service-account rotate on a schedule — the secret only ever lived in your CI secret store.

Step 3 — A branch-aware symbol table

Create the table before the handler that fills it. One row per (branch, path), merge-keyed on a synthetic entry_id (repo@branch:path) so the same file on different branches stays distinct, while each row's s3_key points at the commit-addressed blob shared across branches. Because the table is merge-keyed, the sync's writes are idempotent upserts — re-runs and shard retries are safe. The branch column is what lets a search scope to one branch or span them all.

You

Create a symbols table in devbrain keyed on entry_id, with repo, branch, path, symbol, kind, lang, commit, and s3_key.

DODIL MCP tools called
data_table_create
Agent

Created the symbols table with PRIMARY KEY entry_id (repo@branch:path) and a branch column (column_count 9). The sync handler (next step) upserts one row per (branch, path) — so a file on three branches is three rows pointing at the same commit-addressed blob.

Step 4 — Sync GitHub → DataK3 (the centerpiece)

Deploy an Ignite function as the webhook target. It handles three GitHub events: a push (route changed source to repos/ and ADRs to notes/, and upsert symbol rows), a branch created (seed the new branch from its base), and a pull request (capture the description to notes/ — the why). Source objects are keyed by commit, not branch, so a file identical across branches is stored and embedded once.

# sync.py (essence) — one Ignite function, three GitHub event paths.
# Headless auth via DODIL_SERVICE_ACCOUNT_ID / DODIL_SERVICE_ACCOUNT_SECRET.
def handle(event):
    repo = event["repository"]
 
    # PULL REQUEST — capture the "why": write the PR description under notes/ (embeds into notes_docs).
    if event["kind"] == "pull_request":
        data_put(BUCKET, f"notes/{repo}/pr-{event['number']}.md", event["body"])
        return
 
    # NEW BRANCH — a create event changes no files; seed the branch by copying the base
    #   branch's rows (same commit-pinned objects). No re-upload, no re-embed.
    if event["kind"] == "branch_created":
        for r in sql(f"SELECT path, commit, s3_key FROM symbols "
                     f"WHERE repo='{repo}' AND branch='{event['base_branch']}'"):
            upsert_symbol(repo, event["branch"], r.path, r.commit, r.s3_key)
        return
 
    # PUSH — route source to repos/ (embeds into dev_code) and ADRs to notes/ (into notes_docs).
    branch, commit = event["branch"], event["after"]
    for path in event["changed_files"]:
        if is_design_doc(path):                                 # e.g. docs/adr/*.md, **/adr-*.md
            data_put(BUCKET, f"notes/{repo}/{path}", github_raw(repo, commit, path))
            continue                                            # the "why" — embedded, not a symbol
        key = f"repos/{repo}/{commit}/{path}"                   # commit-addressed source -> dev_code
        if not data_exists(BUCKET, key):
            data_put(BUCKET, key, github_raw(repo, commit, path)) # code-rag embeds it once
        upsert_symbol(repo, branch, path, commit, key)          # one row per (branch, path)
    for path in event["deleted_files"]:
        delete_symbol(repo, branch, path)

So the handler populates all three pillars: the symbols table (Step 3), the dev_code embeddings (source under repos/), and the notes_docs embeddings (ADRs + PR descriptions under notes/). On a push it upserts the changed files; on a new branch it copies the base branch's rows, so the branch is searchable the instant it exists — no re-upload, no re-embed. (Cut a branch from somewhere other than the default? The seed uses your base, and the first real push corrects whatever diverged — shared files still dedup by commit.)

You

Deploy my sync function to Ignite as a public webhook target and give me its URL so I can register it as a GitHub push/PR webhook.

DODIL MCP tools called
ignite_app_deploy
Agent

Deployed sync (deployment_state deployed, public_invoke true) — URL https://sync-ihdiash-8080.ignite.dodil.cloud. Add it as a GitHub webhook (events: push, create, pull_request). Pushes route source to repos/ (→ dev_code) and ADRs to notes/ (→ notes_docs) and upsert symbol rows; PRs write their description to notes/; a new branch seeds instantly from its base.

Step 5 — Search "where + why", across every branch

Now any developer's agent can ask the org-wide question. SQL narrows to the right symbols; Vector finds the code by intent in dev_code and the why in notes_docs; and the object store quotes the exact commit-pinned file and ADR. data search embeds the query server-side and runs hybrid retrieval over a recipe collection — scope it to one collection with --collection, drop weak hits with --min-score.

You

In devbrain, find the code that tunnels NATS traffic and reconnects (by intent, across all active branches), explain why the ping interval is 15s, and link the exact file on feature/tunnel-hardening and the ADR.

DODIL MCP tools called
data_sql→data_search→data_object_url
Agent

Code (dev_code, by intent): copy_bidirectional in tunnel.rs — on main, feature/tunnel-hardening, and release/1.8 (3 branches, commit 3f9a1c). Why (notes_docs): ADR-0007 — the 15s ping is half the broker's 30s idle timeout, so a reconnect fires before the client is dropped. Links (valid 1h): …/dodil-ignite/3f9a1c/…/tunnel.rs?X-K3-Token=… · …/notes/adr-0007…?X-K3-Token=…

Step 6 — Share work in progress: push a draft branch

No need to sync a laptop's uncommitted files. To share pre-review work, a developer pushes a draft branch — the webhook syncs it like any other, and a teammate's agent can search it seconds later. Freshly upserted symbol rows are visible to reads immediately (read-your-writes).

You

On devbrain, what's the new retry logic Sara is working on — search her draft branch spike/retry-backoff.

DODIL MCP tools called
data_sql→data_search→data_object_url
Agent

On spike/retry-backoff: with_backoff in retry.rs (added 2h ago, commit a1b2c3) — exponential backoff + jitter capped at 30s, honoring Retry-After. Not yet on main. Link to the draft file (valid 1h): …/dodil-ignite/a1b2c3/…/retry.rs?X-K3-Token=…

DataK3 vs. bolting it together

ConcernTypical stackDataK3
Code blobs + semantic index + symbol DBObject store + vector DB + Postgres + an indexer serviceOne bucket; a recipe embeds files into a collection, a table holds symbols
Keep it fresh across branchesKafka → stream processor → re-embed → upsert, self-runGitHub webhook → Ignite function → object → rule embeds incrementally
"Find by intent, on this branch, and why"Vector hit joined to a symbol DB, ADRs in a wiki elsewhereSQL narrows, Vector ranks, ADRs are objects next to the code

How this differs from Sourcegraph / Copilot

Semantic code search isn't new — Sourcegraph, Copilot, and Claude Context all do a version. What's different here isn't the search, it's the substrate: the code, the embeddings, the symbol table, the "why" (ADRs), and the sync compute live in one bucket on one platform, one bill — and every piece is agent-native over the MCP, so your agents build on it instead of scraping a UI.

Troubleshooting

  • Re-embed cost. Re-embedding on every push adds up — debounce noisy branches, and let the rule embed incrementally (only changed objects) rather than rebuilding.
  • Branch cleanup. When a branch merges or is deleted, tombstone its keys (or expire by prefix) so search doesn't surface dead branches.
  • Secrets hygiene. The service-account secret lives only in your CI/Ignite secret store; rotate it, and never sync .env/credentials — scope the code-rag --folder/rule includes to source and docs.
  • Chunking. The code-rag collection uses the AST-aware code_embedding_index template (tree-sitter), so recall follows functions/symbols, not arbitrary line windows.

Test

Verify the shared index end-to-end: the bucket exists, both recipes installed their collections, the branch-aware symbols table is present, the sync handler is live, and both SQL and Vector return the code and the why. Reads are read-your-writes by default, so freshly synced rows and objects show up immediately — no compaction step needed (data table compact only drains the write-log for scan perf, never for correctness).

export BUCKET=devbrain
 
# 1. The shared bucket exists
dodil data bucket get "$BUCKET"
# expect: bucket "devbrain" with description "Org code knowledge base: all repos + branches"
 
# 2. Both recipes installed their collections
dodil data vector collection list --bucket "$BUCKET"
# expect: dev_code (code-rag, AST-aware, scoped to repos/) and notes_docs (document-rag, scoped to notes/)
 
# 3. The branch-aware symbols table exists, keyed on entry_id with a branch column
dodil data table describe symbols --bucket "$BUCKET"
# expect: columns entry_id, repo, branch, path, symbol, kind, lang, commit, s3_key; merge-key entry_id
 
# 4. The sync handler is deployed (webhook target)
dodil ignite app get sync
# expect: app "sync" deployed, public_invoke true, with a URL to register as the GitHub webhook
 
# 5. SQL narrows to symbols across all active branches
dodil data sql -b "$BUCKET" \
  "SELECT repo, branch, path, symbol FROM symbols WHERE lang='rust' ORDER BY branch LIMIT 5"
# expect: one row per (branch, path) for the synced Rust files — the same file on N branches = N rows
 
# 6. Vector finds the CODE by intent, and the WHY next to it
dodil data search "bidirectional copy loop that tunnels NATS traffic and reconnects on drop" \
  --bucket "$BUCKET" --collection dev_code --top-k 8 --min-score 0.25
# expect: a hit in dev_code (e.g. copy_bidirectional in tunnel.rs) with a commit-addressed s3_key
 
dodil data search "why is the NATS client ping interval 15 seconds" \
  --bucket "$BUCKET" --collection notes_docs --top-k 8 --min-score 0.25
# expect: a hit in notes_docs (the ADR explaining the 15s ping vs. broker idle timeout)

One-shot

Hand this to an agent with the DODIL MCP connected to reproduce the whole tutorial end-to-end:

Build a shared GitHub → DataK3 code knowledge base in DODIL. Steps:
 
1. Create a DataK3 bucket "devbrain" (data_bucket_create). Install two recipes into it
   (data_recipe_install): code-rag with name-prefix "dev" scoped to folder repos/ (collection dev_code,
   AST-aware), and document-rag with name-prefix "notes" scoped to folder notes/ (collection notes_docs).
   Each recipe stands up a collection + ingest rule — no separate vector store.
2. Create a service account "ci-code-sync" (auth_service-account_create) and grant it the least-privilege
   write role k3-authorization-service/k3.editor (auth_service-account_grant-role). Store its id/secret as
   DODIL_SERVICE_ACCOUNT_ID / DODIL_SERVICE_ACCOUNT_SECRET for the handler — never inline the secret.
3. Create a "symbols" table in devbrain (data_table_create), merge-keyed on entry_id (repo@branch:path),
   with columns entry_id, repo, branch, path, symbol, kind, lang, commit, s3_key.
4. Deploy an Ignite function "sync" (ignite_app_deploy, --allow-unauthenticated) that handles GitHub push,
   create (branch), and pull_request events: route source to repos/<repo>/<commit>/<path>
   (commit-addressed, embeds into dev_code) and ADRs/PR bodies to notes/ (embeds into notes_docs), and
   upsert one symbols row per (branch, path). On a new branch, seed rows from the base branch. Register its
   invoke URL as the GitHub webhook (events: push, create, pull_request).
5. Answer "where + why, on any branch": use data_sql to narrow symbols, data_search over dev_code for
   code-by-intent and over notes_docs for the "why", and data_object_url for a pre-signed link to the
   exact commit-pinned file and ADR.
 
Then run the ## Test checks to confirm the bucket, both collections, the symbols table, the sync app,
and both SQL + Vector results are all present.

Ship it

Step 4 already deploys the sync engine as a public webhook target — that one command is the ship:

dodil ignite app deploy sync --code ./sync --runtime python --tier small \
  --allow-unauthenticated --port 8080 --health-path /healthz

For the full production lifecycle behind it — DODIL git → CI → a scanned image in the registry → a public endpoint, with versioning and one-command rollback — see the flagship walkthrough: Ship a DODIL App.

Conclusion

Every repo and branch, streamed from GitHub into one bucket, intent-searchable with the why attached — a shared code knowledge base a whole team's agents query, always current. One bucket, two recipes, a webhook, and a service account.

Next steps: