JAVIS Technologies
Schema-Driven Ingestion Compiler
A multi-tenant ingestion platform that compiles configuration into executable profiles
95% less code, zero per-client branches
Built with
JavaSpring BootTemporalMySQL
The hard cases move into the compiler. Every client after that is configuration.
Column mappingsValidation rulesIndustry overrides
Compiler
One immutable profile
- The problem
- Every new client meant new ingestion code. Column mappings, validation rules, and industry-specific quirks were expressed as branching logic, so the codebase grew linearly with the customer list and no two onboardings looked alike.
- My role
- Led the design. Owned the compiler, the profile format, and the execution model end to end.
- Approach
- Treated onboarding as a compilation problem rather than a coding one. Column metadata, versioned rulesets, and per-tenant and per-industry overrides compile into a single immutable execution profile, which a generic engine interprets. The cost is that the hard cases move into the compiler, where they are harder to write; the gain is that every client after that is a configuration change.
- Outcome
- Eliminated per-client code entirely and reduced the codebase 95% — from roughly 12,000 lines to 600. Onboarding a client stopped being an engineering task.
What I built
- A compiler for client onboarding. Each client’s column mappings, validation rules, and industry-specific overrides compile into one execution profile, instead of being written as code.
- One engine that runs every client. Built on Temporal, it resumes mid-file after a crash and can replay any past run exactly.
- Speed at scale. Independent lookups run in parallel and repeated work is batched, so a million-row file takes a handful of queries per table, not a million.
Questions about this one? Email me, or ask my AI assistant.