Multi-Lane Speed Measurement Edge Module with License Plate Recognition
Abstract
This project presents a multi-lane speed measurement edge module with real-time license plate recognition (LPR), tailored for Sri Lankan road conditions where lane discipline is not strictly followed. The system uses a 77 GHz FMCW radar sensor (TI IWR1843BOOST) for vehicle speed and position estimation, and a two-stage deep learning pipeline — license plate detection (LPD) followed by license plate character recognition (LPCR) — implemented on a Xilinx Kria KV260 FPGA-SoC. Low-bitwidth quantization (Brevitas) and FPGA acceleration (FINN framework) enable real-time edge deployment at 11.5 FPS. A data fusion module matches radar-detected vehicle speeds with the corresponding license plate, enabling fully automated speed violation capture without additional manpower.
Supervisors
Dr. Sampath Perera · Eng. A. T. L. K. Samarasinghe
Industry Collaborators
PEX Microsystems · National Council for Road Safety (Ministry of Transport and Highways)
Team Members
Key Performance Highlights
System Architecture
The system is built around three tightly integrated subsystems implemented on the Xilinx Kria KV260 SoC: the License Plate Recognition pipeline, the Radar Speed Estimation module, and the Data Fusion engine. Images are captured at 10 FPS via a MIPI CSI-2 camera and processed on the FPGA Programmable Logic (PL) using custom FINN-compiled accelerators. The 77 GHz radar delivers speed, range, and angle data via SPI to the ARM PS. A hardware data fusion module in the PL projects radar detections onto the image plane using a calibrated camera model and matches them to detected license plates by Manhattan distance.
License Plate Recognition Pipeline
License Plate Detection (LPD)
A lightweight YOLO-based model (adapted from a face-detection architecture) was trained for Sri Lankan multi-lane scenarios. The model contains only convolution and max-pooling layers with 4-bit quantized weights and activations, totalling 0.351 M parameters and 0.363 B operations. Input images are resized to 576×576. Custom anchor boxes were computed via K-means clustering on the SL-LPR dataset to better fit the aspect ratios of Sri Lankan plates. A novel 8-bit quantized sigmoid activation (replacing HardTanh) reduced max bounding-box pixel error from 15 to 3, boosting mAP from 52% to 92.5%. Final mAP on SL-LPR: 93.6%.
License Plate Character Recognition (LPCR)
Based on the fast-plate-ocr CNN architecture, restructured and redesigned to be compatible with the FINN FPGA compiler. Mixed-precision quantization (4-bit first 3 layers, 2-bit next 7 layers, 1-bit final layer) and model pruning reduced parameters from 11.47 M to 1.02 M (a 91% reduction) with only a small accuracy drop (89.70% to 87.88%). An intensity normalisation step applied per-plate compensates for extreme lighting conditions (late evening / noon), improving accuracy by 5% under low-light conditions. Final plate-level accuracy: 87.88% · Character-level: 97.42%.
FINN / Brevitas Hardware Acceleration
Both models are compiled from Brevitas-trained ONNX graphs using the Xilinx FINN framework into custom RTL/HLS IP blocks, connected via AXI-Stream and AXI-DMA to the Zynq PS. Multiple optimisation passes in the FINN compilation (SIMD/PE folding, RTL MVAU layers, distributed RAM reduction, separate thresholding layers) reduced LUT usage from ~34K to ~14K while sustaining 55+ FPS for the LPCR model alone. Both models run in a cascade pipeline delivering 11.5 FPS end-to-end.
SL-LPR Dataset
A custom Sri Lankan License Plate Recognition (SL-LPR) dataset was created by capturing video from pedestrian overpasses on urban roads and expressways. A YOLOv10 vehicle-counting model was used to extract non-duplicate frames. License plates were manually annotated using a custom Python tool and labelled via easyOCR followed by manual correction. The final dataset contains 2970 multi-lane images with up to 8 license plates per scene, covering cars, motorbikes, buses, trucks, three-wheelers, and both daytime and low-light conditions. An 80:18:2 train/val/test split was used for LPD training; the LPCR subset contains 3412 license plate crops.
Radar-based Speed Estimation
The TI IWR1843BOOST 77 GHz FMCW radar module is used for speed and position estimation. Chirp parameters were tuned to achieve a velocity resolution of 2.27 km/h (≤ 3 km/h requirement) and a maximum detectable range of 30.6 m. A GTRACK clustering and tracking algorithm groups the detected point cloud into vehicle targets and outputs a list of positions and velocities, transmitted to the Zynq PS via SPI (configured as SPI master) in 64-byte chunks. MATLAB simulations were used to verify radar behaviour before hardware acquisition.
Data Fusion
The data fusion module is implemented as a custom SystemVerilog IP in the FPGA PL. It receives time-synchronised radar and image frames via AXI-Stream DMA. The radar 3D positions are projected onto the image plane using a calibrated camera matrix (intrinsic + extrinsic parameters). A Z-coordinate adjustment compensates for the 2D radar firmware assumption. License plates and radar targets are matched using the Manhattan distance; the pair with the minimum distance below a configurable threshold is accepted as a match. Matching accuracy: 97.8% on normal roads, 96.2% on busy roads, 94.1% on expressways.
Hardware Platform
The Xilinx Kria KV260 Vision AI Starter Kit (Zynq Ultrascale+ MPSoC) serves as the processing platform. Its quad-core ARM Cortex-A53 PS handles radar data parsing, time synchronisation, NMS post-processing, and LPCR output decoding, while all inference accelerators and the data fusion module run in the PL at 100 MHz (camera pipeline at 250 MHz). The complete system consumes 4.2 W and utilises 63% LUTs, 38% FFs, 99% BRAM, and 29% DSPs of the KV260.
Full Results Summary
| Metric | Value |
|---|---|
| LPD mAP (IoU ≥ 0.5) | 93.6% |
| LPD Recall | 92.2% |
| LPCR Plate-Level Accuracy | 87.88% |
| LPCR Character-Level Accuracy | 97.42% |
| System FPS | 11.5 FPS |
| Max Latency per Frame (10 LPs) | 87 ms |
| Velocity Resolution | 2.27 km/h |
| Max Detectable Velocity | 145 km/h (extended to ~290 by firmware) |
| Max Detectable Range | 30.6 m |
| Range Resolution | 0.597 m |
| Power Consumption | 4.2 W |
| LUT Utilization | 63% |
| BRAM Utilization | 99% |
| LPD Parameters | 0.351 M |
| LPCR Parameters | 1.02 M (after pruning, down from 11.47 M) |
| Total Model Size | ~1.4 M parameters, 268 KB after quantization (both models) |
My Contributions
- Conducted literature review on LPR, vision-based speed estimation, and vehicle detection; verified existing methods by running reference code
- Fine-tuned the LPD model (LPYOLO) and trained it on the custom Sri Lankan SL-LPR dataset
- Implemented input image resizing for the LPD model in C (optimised for efficiency) and Halide language
- Implemented Non-Maximum Suppression (NMS) in C
- Developed input image resizing/cropping from LPD detections for the LPCR model
- Integrated the full end-to-end pipeline and displayed outputs on monitor
- Designed enclosures for field testing and the final prototype
- Contributed to field testing, real-time camera integration, and documentation
Related Publication
This project led to the ITSC 2026 paper: "An Embedded Real-Time License Plate Recognition System for Complex Traffic Scenes".
