Broker's laptop
Full extraction pipeline on a standard CPU — batch a day's B/Ls over lunch.
Open-source Logistics NER
Your Data. Your Model. Your Hardware.
Extract 30+ trade-document entities from bills of lading, invoices & customs forms — 100% on your hardware.
Eight trained encoder and document models that read B/Ls, air waybills, commercial invoices, packing lists and customs declarations, validate the codes that matter (ISO 6346, IMO, HS, UN/LOCODE), and de-identify pricing & PII on-prem with a signed audit trail — no document ever leaves your network. ONNX/edge exports included. Apache-2.0, hosted on Hugging Face.
8
models trained
9
languages
~150 ms/doc
on-device
$0 · 0 bytes
cost · egress
Quickstart
Four lines to structured trade data
- Profiles, not pipelines. customs, ocean, air, redact — each bundles the right models and code lists.
- Validated, not just extracted. Container check digits, HS chapter structure and UN/LOCODEs are verified before anything is returned.
- Structured output. Entities, spans, confidences — plus EDIFACT/X12 export for downstream systems.
$ pip install git+https://github.com/flowx-ai/openner
from openner import analyze result = analyze(document, profile="customs") print(result.entities) # CONTAINER_NO, HS_CODE, PORT, INCOTERM … edi = result.to_edifact() # structured output, validated on-device
Runtime
Runs everywhere your documents live
The largest model is under 400 MB. No GPU, no cloud account, no callbacks.
Port & warehouse edge
Gate-in OCR and container validation on an edge gateway, air-gapped if you like.
Inside your TMS / ERP
Embed as a Python or Java library — extraction becomes just another function call.
Browser (ONNX / WASM)
Quantized ONNX builds run client-side — nothing uploads, even in a web app.
REST / gRPC service
One container image serves your whole team behind your own firewall.
Privacy & de-identification
Data sovereignty for trade data
Rates, supplier lists and routings are your competitive edge. Customs data is regulated. OpenFreight is built so neither has to leave the building — even when you want an LLM downstream.
TradeRedact before any LLMStrips negotiated prices, supplier identities, banking details and personal data before a document ever reaches a cloud or LLM call.
Check-digit validationISO 6346 container numbers, IMO vessel numbers and GTIN/SSCC identifiers are verified mathematically, not just pattern-matched.
Code-list gazetteersShips with the HS nomenclature, UN/LOCODE directory and SCAC registry baked in — entities resolve to canonical codes, offline.
Layout-aware extractionUnderstands the geometry of scanned and faxed documents — a value in the "Consignee" box is a consignee, wherever the box sits.
Export-control & DG flagsUN numbers, IMDG/IATA classes and dual-use keywords raise flags early — before a shipment becomes a compliance incident.
Zero data movementNo telemetry, no phone-home, no usage analytics. The only network call OpenFreight ever makes is the one-time model download.
Model library
Small models for trade documents
Each model does one job well, stays small enough for a CPU, and is published on Hugging Face under Apache-2.0. Compose them through profiles or call them individually.
ContainerDetect
Extracts and ISO-6346-validates container numbers, seals and size/type codes.
CONTAINER_NOSEALSIZE_TYPE
HSCodeClassify
Classifies a goods description into its HS chapter (~20 chapters) for chapter-level customs routing — trained on a synthetic factory; retrain on real CBP CROSS rulings before it drives binding tariff determinations.
HS_CHAPTERCOMMODITY
IncotermExtract
Detects Incoterms 2020 rules and the named place that goes with them.
INCOTERMPLACE
BoLParse
Full bill-of-lading field extraction, from shipper block to cargo description.
SHIPPERCONSIGNEEPORTCARGO
PortRoute
Resolves UN/LOCODEs, vessel and IMO numbers, voyage references, ETA/ETD.
UNLOCODEIMOETA
HazmatGuard
Flags UN numbers, IMDG/IATA hazard classes and packing groups.
UN_NOIMDG_CLASSPACKING_GRP
PartyLink
Disambiguates shipper, consignee, notify party and carrier — with SCAC lookup.
SHIPPERCONSIGNEESCAC
TradeRedact
PII and commercial de-identification: names, contacts, banking, unit prices.
NAMEPRICEBANKCONTACT
Built with OpenFreight
From toolkit to product
The models are the foundation. These are the applications teams build on top — all of them running on-prem, all of them open.
Customs-Clearance Copilot
Reads the B/L, commercial invoice and packing list together, drafts the customs declaration, proposes HS codes with a written rationale, and flags export-control and dangerous-goods risk — entirely on-prem.
Doc-to-EDI Digitizer
Turns scanned and emailed shipping documents into structured EDIFACT and X12 messages, with every code validated against the official lists before it hits your message queue.
Shipment Data Vault
Locally indexes your whole document archive and answers questions like "where is PO 44821, what's the ETA, any hazmat on board?" — de-identifying with TradeRedact before any LLM call.
Standards
Speaks the language of trade
Extraction is only useful if it lands in the frameworks your compliance team already works in.
WCO / AEOtrusted-trader programmesC-TPATsupply-chain securityIMDG / IATA DGRdangerous goodsGDPRpersonal data in documentsITAR / EARexport-controlled goodsGS1GTIN / SSCC identifiersUN/EDIFACTcustoms message syntaxUN/LOCODEports & locationsHS Conventiontariff classification
Benchmarks
Measured against frontier LLMs
We benchmarked our small on-device models head-to-head against the latest cost-effective frontier LLMs — same documents, same entity types. N=30 per task, against Claude Haiku 4.5, GPT-5.4-mini, GPT-5.4-nano, Gemini 3.5-flash and Gemini 3.1-flash-lite (OpenAI models via the Responses API).
| Extraction — invoices · PII · trade fields | F1 | Latency / doc | Cost / 1k docs | Egress |
|---|---|---|---|---|
| OpenFreight(8 models, on-device) | 1.000 | ~150 ms | $0 | 0 bytes |
| Frontier tier(Haiku 4.5 · GPT-5.4-mini/nano · Gemini 3.5-flash / 3.1-flash-lite) | ≤ 1.000 | 850–3,700 ms | $0.03–0.84 | full document |
6–24× faster
~150 ms per document on a CPU, versus 850–3,700 ms round-trips to frontier APIs.
$0 vs $0.03–0.84
Per 1,000 documents. Once the weights are downloaded, marginal inference cost is zero.
0 bytes
Leave your network. No API calls, no telemetry — rates, parties and PII stay in the building.
Honest footnote: frontier LLMs win at open-ended reasoning — not structured, convention-bound extraction. That's where small on-device encoders win.
FAQ
Questions, answered plainly
Q.01What is OpenFreight?
An open-source toolkit of small, task-specific NER and document-understanding models for logistics and trade paperwork — bills of lading, air waybills, commercial invoices, packing lists and customs declarations. It extracts and validates the entities those documents carry, and de-identifies the sensitive ones, entirely on your own hardware.
Q.02Why small encoder and document models instead of a big LLM?
Encoder models (BERT-family) are excellent at token classification, run in milliseconds on a CPU, produce deterministic spans with confidences, and don't hallucinate container numbers. For a well-defined extraction task on regulated documents, a 100–400 MB model you can audit beats a remote multi-billion-parameter one you can't. LLMs still have a place — downstream, after TradeRedact.
Q.03Does my trade data ever leave my network?
No. Inference is fully local, there is no telemetry, and the library makes no network calls at runtime. The only download is the model weights themselves, once, from Hugging Face — and you can mirror those internally for air-gapped sites.
Q.04How does it handle scanned and faxed documents?
Two paths: layout-aware models in the LayoutLMv3 family that combine OCR text with page geometry, and OCR-free Donut-style models that read the page image directly. Profiles pick the right path automatically based on whether the input is digital text or an image/PDF scan. Our DocFormNER model (LayoutLMv3: text + 2D layout + page image, published on Hugging Face) now reads scanned and photographed bills of lading and arrival notices on-prem — it's a preview trained on synthetic rendered forms, so validate it on your own OCR'd scans before production.
Q.05Can I fine-tune on my own documents?
Yes — that's the intended workflow for house formats. Every model ships with its training configuration, and the openfreight.finetune module wraps Hugging Face Transformers so a few hundred annotated examples of your own B/L layout are usually enough to specialize a model. Your fine-tuned weights stay yours.
Q.06Is it really open source? And what is it trained on?
Apache-2.0 across code, weights and training recipes — commercial use included. On data, an honest note: there is no large public logistics-NER corpus. Training combines public document-KIE datasets (DocILE, SROIE, CORD, FUNSD), official trade code lists (CBP CROSS rulings, the HS nomenclature, UN/LOCODE, the BIC container register) and large-scale synthetic document generation. Dataset cards document the mix per model.
openfreight.life — an open-source concept. Code, weights and recipes under Apache-2.0. Modeled on the OpenMed approach (openmed.life).