Operate the mastering lane.

First installation

Prerequisites: Git, Docker with Compose v2, 4 GB free memory, and ports 8088, 55432, and 58080. Clone the platform and run:

git clone https://github.com/dlamaro96/recordlane.git
cd recordlane
./recordlane doctor
./recordlane demo
open http://127.0.0.1:8088

The demo binds host ports to loopback, generates synthetic data only, and disables arbitrary destinations. Use ./recordlane status and ./recordlane logs for diagnosis.

First mastered supplier

Open Source records to see original English, Spanish, and Arabic contributions. Quarantined records remain evidence. Open Match workbench to decide an ambiguous pair, then use Steward inbox as a distinct approver to publish an approved master. The master page explains each selected attribute.

Modeling

Domain packs use recordlane.io/v1alpha1, declare an operating mode, typed attributes, identifier namespaces, relationships, validation, matching, and survivorship. Supplier, customer, product, asset, location, and organization examples live in the ecosystem repository. Unsupported behavioral settings are rejected. Treat a change as a versioned draft, run the exact impact simulator, and obtain independent approval; activation queues durable remastering against the same compiled policy.

Connector setup

A manifest must declare direction, formats/entities, full and incremental reads, deletions, discovery, auth, write operations, rate limits, consistency, checkpoints, compatibility, and actual validation status. Production administrators allow destinations; connector secrets never enter domain configuration.

Snapshot and delta behavior

Complete a bounded full snapshot before switching to a durable source position. Timestamp polling uses a stable key tie-breaker plus bounded overlap and ingestion deduplication. Failed or incomplete pages never imply deletion. Pagination loops and non-advancing checkpoints stop the job.

Quality rules

Normalization is deterministic Unicode NFKC, case folding, and whitespace handling. Validation failures retain the original payload in quarantine. Profiling states population and scope; the UI does not invent a score when a denominator is absent.

Matching and thresholds

Blocking constrains candidates. Weighted name/email/country evidence drives an explicitly labeled score, not a probability. Conflicting verified tax or registration identifiers are hard constraints across the whole cluster. Auto-link and review thresholds are separate and simulated before policy activation.

Survivorship

Selection is attribute-specific: verification state, source priority, then observation time. Approved values and candidate values are distinct. Provenance includes source record/version and the rule that selected the value.

Approvals

Tasks bind entity and configuration versions. The creator cannot approve their own governed change. Any source or master version change invalidates stale approval work. Rejections and comments are durable audit facts.

Merge, split, and downstream repair

Preview impacted source contributions, aliases, relationships, policies, and consumers. An approved merge or split preserves evidence and history, writes a correction event, and opens downstream repair work. Local correctness never claims external rollback.

Relationships

Relationships are typed and effective-dated, enforce modeled cardinality in policy, and have both a graph and accessible table. Identity correction rewrites references transactionally.

Configuration as code

Validate YAML against the published schema, prohibit inline secret-like keys, store a canonical checksum, simulate against a data checkpoint, and require review before activation. The UI and files drive the same backend versions.

SDKs

The Python and TypeScript SDKs expose capabilities, domains, entities, tasks, decisions, publication, pagination, timeouts, credentials, typed errors, and safe retry behavior. Run their live tests against this exact release.

Service accounts

Create a workspace-scoped service identity with least-privilege roles and scopes. Recordlane returns its credential once and stores only a verifier; rotate by issuing a replacement and revoking the old account. Issuer-managed OAuth clients remain supported through normal bearer validation. Never reuse user cookies or connector credentials.

OIDC and brokered SAML

The browser uses authorization code with PKCE, state, and nonce. Tokens remain encrypted in server-side sessions and the browser receives an HttpOnly SameSite cookie. Production validates issuer, audience, lifetime, signature, workspace membership, and roles. Bundled Keycloak provides the real local flow and can broker SAML upstream; Recordlane itself does not implement SAML.

Supported SCIM operations

The SCIM 2.0 surface supports users, groups, membership updates, filters, pagination, deactivation, and session revocation under a dedicated provisioning credential. Unsupported operations fail explicitly. Rotate the SCIM credential through the deployment secret and test deprovisioning before production use.

Secrets

Configuration stores named references, never values. The open distribution includes mounted environment/file references, encrypted-local storage under a mounted master key, and a Vault adapter. Listing and connection tests do not return plaintext. Startup rejects weak session secrets, missing keys, wildcard CORS, and unsafe production settings.

Publication

Approved master writes and outbox events commit together. Relay is at least once, HMAC-signed, timestamped, and retryable. Consumers deduplicate by event id and reconcile entity versions. A 2xx is acceptance—not proof of business consistency.

Compose production

Copy deploy/compose/.env.production.example, provide PostgreSQL and OIDC endpoints, mount a master key, and run the production profile. It deliberately does not bundle a production database or identity provider. Run preflight before traffic.

Kubernetes

Install the Helm chart with explicit issuer, audience, secrets, ingress, and allowed egress CIDRs. The chart applies non-root, read-only filesystem, dropped capabilities, disruption budgets, network policy, and a migration hook. Run helm lint and inspect rendered manifests before install.

Disconnected deployment

Mirror pinned image digests and chart/archive checksums into the disconnected registry, export the release manifest, disable optional internet connectors, and configure an internal OIDC issuer. The app does not require external AI or telemetry.

Backups

Use the root CLI to create a PostgreSQL custom-format dump plus a manifest of platform/schema versions and counts. Encrypt and store it outside the database host. Back up external object data using the same consistency boundary.

Disaster recovery

Restore into a separately named database with outbound publication disabled. Run migrations/preflight, verify counts, links, configurations, open tasks, and audit-chain continuity, then reconcile the sink by immutable event and entity version before enabling egress. Never guess delivery state.

Upgrade and migrations

Back up first and check the compatibility manifest. Compose runs a one-shot migration dependency; Helm uses a pre-install/pre-upgrade job. The versioned runner acquires a PostgreSQL advisory lock, verifies revision checksums, recognizes the unversioned alpha baseline, and backfills stable source identities without deleting observations. Run smoke tests after it reports schema current. Database downgrade is restore-based unless a migration explicitly documents reversibility.

Observability

Health endpoints distinguish liveness and readiness; correlation IDs connect API failures to logs. Track queue lag, processing lag, source freshness, and publication lag separately. Do not put record values, tokens, or connector secrets in labels/logs.

Troubleshooting

./recordlane doctor checks prerequisites and manifests. Use Compose status/logs, verify readiness, inspect durable job leases, then review outbox attempts and dead letters. A Keycloak outage must produce authentication failure, never demo fallback in production.

Contributor tutorial

Run ./recordlane demo and ./recordlane test. Copy the ecosystem tiny connector, declare its capabilities, add one domain-pack validation, run connector conformance, platform tests, and Playwright, capture the changed real UI state, then open a focused PR with migration and security impact. Core API or matcher changes are not needed for a new connector.

Generated API reference

Download the canonical OpenAPI JSON. The /api/v1/capabilities response is the runtime authority; clients must not assume unavailable behavior.