Certification delays in wireless products almost never come from a single catastrophic failure.
They accumulate from gaps: things that were not tested because the team assumed they would work, because there was no process to catch regressions, or because the certification requirement was not understood until it appeared on a checklist days before the submission window.
The eight gaps below are the most common sources of delay across Bluetooth SIG qualification, Matter CSA certification, Zigbee certification, and regulatory compliance regimes including ETSI EN 303 645 and PSA Certified Level 1. Each gap can add days or weeks to a product launch. In combination, they routinely push wireless testing and certification from a four-week exercise into a four-month one.
Score yourself before reading the individual entries. Mark your honest status for each gap in the table below. More than two “not started” answers indicates material certification risk.
Self-assessment checklist
| # | Testing gap | Key symptom | Your status |
|---|---|---|---|
| 1 | No CI/CD pipeline for wireless firmware | Qualification regressions found at release | ☐ Done ☐ Partial ☐ Not started |
| 2 | No hardware-in-the-loop test bench | RF behavior unknown until field trials | ☐ Done ☐ Partial ☐ Not started |
| 3 | Bluetooth PTS qualification run manually | PTS regression takes 1–2 weeks per cycle | ☐ Done ☐ Partial ☐ Not started |
| 4 | Protocol certification testing started too late | Matter/Thread/Zigbee cert adds 4–8 weeks at launch | ☐ Done ☐ Partial ☐ Not started |
| 5 | OTA update path not validated under failure conditions | OTA fails in field conditions not reproduced in the lab | ☐ Done ☐ Partial ☐ Not started |
| 6 | Power budget measured once, not continuously | Battery-life regressions go undetected between builds | ☐ Done ☐ Partial ☐ Not started |
| 7 | Security testing deferred | EN 303 645 / PSA gaps found at pre-launch compliance review | ☐ Done ☐ Partial ☐ Not started |
| 8 | Multi-protocol interoperability assumed, not tested | Failures on third-party hardware discovered post-launch | ☐ Done ☐ Partial ☐ Not started |
Gap 1: No CI/CD pipeline for wireless firmware
Most embedded teams have some form of automated build: a script that compiles firmware and runs a basic sanity check. Far fewer have a pipeline that runs protocol-layer regression tests on every commit or nightly build. The distinction matters because protocol bugs look different from application bugs: they tend to be subtle, timing-sensitive, and intermittent, which means they are easy to miss in the single regression run that happens before a certification submission and easy to catch in a continuous test process that runs against every change.
The practical consequence of missing CI/CD regression testing at the wireless protocol layer is that specification regressions accumulate silently between releases. A change to connection parameter handling fixes a power consumption issue but introduces a subtle timing violation in the GATT server behavior. The firmware builds correctly, all unit tests pass, and the violation is not discovered until the PTS regression runs two weeks before the certification submission. At that point several other changes have been layered on top and the root cause takes days to isolate.
Building CI/CD for embedded firmware in ci cd pipeline embedded systems terms is harder than for web software. It requires physical test hardware, real radio interfaces, and test orchestration that can drive protocol state machines rather than HTTP requests. Once built, the infrastructure eliminates an entire class of late-cycle surprise. The right approach is to run a curated subset of the most regression-sensitive protocol tests on every commit, and the full suite nightly or on every release candidate.
Gap 2: No hardware-in-the-loop test bench and why HIL and RF test bench setup cannot be deferred
Wireless firmware interacts with hardware in ways that simulation cannot fully reproduce. The RF front end, antenna matching network, crystal oscillator tolerance, and power supply noise all affect protocol behavior, not dramatically under ideal conditions but measurably under real ones. A HIL test bench puts real firmware on real hardware and drives it through test cases using automated stimuli, producing results that reflect actual device behavior rather than a model of it.
Teams without a HIL bench rely on a combination of software emulation and ad-hoc manual testing with development kits. This catches the majority of functional bugs but misses two important categories: bugs that appear only on production hardware, where the RF environment, antenna, and power supply differ from the development kit; and bugs that appear only under specific RF conditions, such as marginal signal, 2.4 GHz interference from co-located radios, or coexistence with other devices sharing the band. An RF test bench adds the controlled RF environment that development kit testing lacks (attenuators, shielded enclosures, calibrated signal sources), making it possible to reproduce the marginal-signal conditions that produce intermittent field failures.
For UWB ranging products specifically, accuracy claims (±10 cm, ±30 cm, sub-meter) can only be validated against real hardware in a controlled RF environment. UWB timing accuracy is sensitive to antenna placement, cable delays, and clock synchronization in ways that make software-only testing meaningless for ranging accuracy verification. The same principle applies to any wireless product whose specification contains RF performance claims rather than purely functional ones.
Building a HIL test bench requires hardware design, firmware support for a test control interface, and software automation in the orchestrator. It is a non-trivial upfront investment that amortizes across years of product development and eliminates the category of “worked in the lab, failed on production hardware” surprises.
Gap 3: Bluetooth PTS qualification run manually
The Bluetooth SIG Profile Tuning Suite is the software tool that drives Bluetooth qualification testing. Running PTS manually means an engineer sits in front of a Windows PC, selects test cases one at a time, configures the device under test for each case, interprets the results (which range from clear pass/fail to genuinely ambiguous inconclusive outcomes), and repeats for as many as several hundred test cases across the profiles the device supports.
A full manual PTS regression for a non-trivial BLE device takes one to two weeks of a senior engineer’s time. Most teams run this regression at most once per release, which means any bluetooth certification testing regression introduced during development is discovered late, debugged under time pressure, and potentially blocked if the fix requires retesting a large portion of the qualification matrix. Automated bluetooth testing at the PTS level means the same regression runs overnight without a human in the loop.
PTS automation replaces the human operator with a script. The script drives PTS through its Windows UI using UI automation frameworks, configures the device under test through a serial API, captures the result of each test case, and loops through the full matrix without human intervention. A regression that previously took two weeks runs overnight and produces a complete result log before anyone arrives in the morning. The specification-level bugs that automated PTS catches are real certification risks: Bluetooth SIG submission reviewers examine PTS logs, and failures caught after submission can result in a rejected qualification requiring a second cycle, typically adding four to six weeks to the timeline.
Gap 4: Protocol certification testing started too late
Matter, Thread, Zigbee, and Bluetooth certification are each multi-step processes with formal submission gates. Matter certification requires testing through a CSA Authorized Test Provider, producing a Certification Declaration blob and an entry in the Distributed Compliance Ledger. The Bluetooth SIG qualification process produces a Qualified Design Listing entry. Zigbee certification testing involves equivalent formal review steps through the Connectivity Standards Alliance, and teams that have not run Zigbee conformance tests before submission commonly discover profile-level failures that add two to four weeks per iteration.
Each process takes time: typically four to eight weeks from submission to outcome, assuming the first submission passes. A failed submission cycle (caused by conformance test failures, incomplete test evidence, or incorrect PICS/PIXIT declarations) adds another four to eight weeks for resubmission.
Teams that treat certification as a final step, something to start when the firmware is otherwise complete, are building a best-case timeline of four to eight weeks into their launch date plus the implicit risk of a failed submission. Teams that start pre-certification testing three to six months before launch have time to discover and fix conformance gaps before the formal submission window, iterate on the PICS/PIXIT declarations, and engage with the certification body on ambiguous test results without deadline pressure.
The specific mechanism for starting early is running the public test suites independently. Matter’s Python-based conformance tests and the chip-cert toolchain can be run against the device before any CSA Authorized Test Provider is involved. The PTS tool is publicly downloadable. Running these tools at feature-complete firmware (or even at prototype stage) surfaces the gaps early, when they are cheapest to fix.
Gap 5: OTA testing services absent and the update path not validated under failure conditions
Over-the-air firmware updates are a standard feature of connected devices, but the OTA path is among the least thoroughly tested aspects of most firmware projects. Functional testing covers the happy path: device receives the update, validates it, applies it, reboots into new firmware, confirms success. What usually goes untested is the failure path: partial write, power loss during transfer, corrupted image, rollback failure, network interruption mid-transfer.
OTA testing services that cover only the happy path are not OTA testing. They are OTA smoke testing. The failure modes that matter are the ones that produce bricked devices or insecure rollbacks: a corrupted block at byte 60,000 of a 64,000-byte transfer, a power interruption at the moment of the flash write, a watchdog reset during the integrity check. These are the scenarios that produce customer escalations and field returns.
OTA failure scenarios matter for certification because several regulatory frameworks imply that the update mechanism must be reliable. ETSI EN 303 645 provision 5.3 requires that software updates can be applied in a timely manner without degrading security configuration. A device that bricks on a failed OTA update fails this requirement. A device that falls back to an insecure previous firmware version on OTA failure may also fail it, depending on the interpretation.
Beyond regulatory compliance, OTA failures in the field are expensive in a different way from development bugs. They require hardware returns or field dispatch, create customer trust failures, and can leave devices in states from which recovery requires physical access. Testing the failure path during development (simulating interrupted transfers, corrupt images, and power loss at various stages of the update sequence) is the only reliable way to know how the device behaves under those conditions before they occur in production.
Gap 6: Power budget measured once, not continuously
Battery-powered wireless devices ship with power consumption specifications: current draw in sleep, in connected idle, in active advertising or ranging. These are typically validated once, on the hardware platform and firmware build that existed when the power characterization was done. They are rarely re-validated continuously as firmware evolves.
The risk is that firmware changes affect power consumption in ways that are not obvious from code review. A new feature adds a timer wake-up that fires slightly more often than intended. A protocol stack update changes advertising behavior in a way that increases on-air time by 10%. A connection parameter negotiation change keeps the radio active during connection intervals that were previously sleeping. Each change may be small enough to pass manual review, but their cumulative effect can push a product outside its rated battery life by the time it reaches certification.
IoT power consumption testing integrated into CI means running a standardized power profile measurement on each firmware build and comparing against a baseline. The measurement hardware is straightforward: a power analyzer connected between supply and device, with automated current logging through the test run. The orchestration integrates with the same CI pipeline that runs protocol regression tests. Products targeting PSA Certified Level 1 or EN 303 645 compliance should note that while neither standard requires power testing directly, the broader engineering discipline that certification-focused development demands implicitly raises the bar for regression testing across all product parameters.
Gap 7: IoT device security testing deferred
ETSI EN 303 645 defines 68 security provisions for consumer IoT devices, of which 33 are mandatory. The European Union’s Cyber Resilience Act, the UK Product Security and Telecommunications Infrastructure Act, and equivalent frameworks in other markets reference or align with EN 303 645, making it the de facto baseline for IoT device security testing across major markets.
Deferred security testing, addressed at the end of development or prompted by a customer’s security questionnaire, tends to produce surprises that are expensive to fix at that stage. The most common findings in IoT device security testing involve hardcoded credentials (provision 5.1), absence of a vulnerability disclosure policy (provision 5.2), insecure default configurations (provision 5.4), and unencrypted sensitive communications (provision 5.8). These are architectural issues, not configuration switches, and fixing them after the product is otherwise feature-complete typically means rework to core firmware modules.
PSA Certified Level 1 is an industry certification for IoT device security aligned with EN 303 645 and NIST SP 800-213. It provides a structured framework for assessing security posture earlier in the development cycle. The Level 1 assessment covers device identity, secure boot, isolation, and update mechanisms through a questionnaire. Working through the PSA Certified questionnaire at the beginning of the development cycle converts the assessment from a compliance checkpoint into a design guide.
For wireless products specifically, the security surface is larger than for wired-only devices. BLE, UWB, Wi-Fi, and Zigbee/Matter each introduce protocol-layer attack surfaces (eavesdropping, replay, man-in-the-middle, denial of service) that require protocol-level security testing beyond application-layer review.
Gap 8: Wireless interoperability testing not planned
Multi-protocol wireless devices (products that combine BLE with UWB, Zigbee with Matter, or BLE with Wi-Fi) are increasingly common. The protocols themselves are well-specified and certified independently. What is rarely tested is how they behave together on the same device, particularly in the 2.4 GHz band where BLE, Zigbee, Wi-Fi, and Thread compete for spectrum.
The failure modes that wireless interoperability testing catches include radio state machine conflicts (the BLE stack requests the radio while the Zigbee stack has it allocated), timing violations under load (high BLE advertising rate reduces Zigbee packet delivery reliability by 30% in high-traffic conditions), and coexistence arbitration failures where the firmware does not correctly handle simultaneous connection events across protocols.
Beyond coexistence, wireless interoperability testing covers compatibility with third-party hardware. A BLE device that pairs correctly with the manufacturer’s own mobile application may fail to pair with third-party applications implementing the same profile. A Matter device that passes conformance testing with the CSA’s reference implementation may fail with specific ecosystem controllers from third-party manufacturers. Finding these failures after launch means firmware updates, customer escalations, and potential recertification cycles.
A realistic interoperability test plan covers: coexistence testing under high-traffic conditions with all supported protocols running simultaneously; compatibility testing against at least two or three well-known third-party implementations for each protocol; and regression verification that coexistence behavior is stable across firmware releases.
needCode specializes in wireless firmware test automation for embedded products, including Bluetooth PTS automation, hardware-in-the-loop bench design, OTA failure-mode validation, and EN 303 645 / PSA Certified pre-assessment. We have built continuous test programmes across BLE mesh, UWB ranging, and multi-protocol embedded engagements for semiconductor vendors and IoT product companies. If your current certification process consists of a manual PTS sprint and a last-minute security audit before launch, we are happy to talk about what closing these gaps with continuous automated coverage would look like for your product.
Book a free discovery call or get in touch
Frequently asked questions
A full assessment against ETSI EN 303 645, using the test procedures in TS 103 701, typically takes four to eight weeks depending on device complexity and the number of mandatory provisions under review. Organizations mapping to PSA Certified Level 1 can use the PSA Certified questionnaire as a structured input to the EN 303 645 assessment, which reduces the time required for the compliance mapping work. Starting the assessment three to four months before a market launch leaves time to fix findings that require firmware changes.
IoT device security testing covers the security provisions defined in ETSI EN 303 645, including credential management, software update security, communication encryption, attack surface minimization, and vulnerability disclosure processes. For wireless products, it also covers protocol-layer security: BLE pairing security modes, UWB session integrity, Zigbee and Matter security architecture, and resistance to common wireless attacks (replay, eavesdropping, denial of service). A pre-certification IoT device security test maps findings to EN 303 645 provisions and to PSA Certified Level 1 requirements, producing a remediation list prioritised by regulatory risk.
Pre-certification testing runs the same test suites that a formal certification body would use, but without the formal submission and without producing the certificates or listings that the official process generates. Its value is finding and fixing failures before the formal submission, so that the first formal submission is likely to pass. Pre-certification testing is generally faster and cheaper to iterate on than re-submission cycles through formal certification bodies.
Yes. The Bluetooth SIG qualification process applies to products claiming conformance with the Bluetooth specification, which covers both Bluetooth Classic (BR/EDR) and Bluetooth Low Energy. Products that support only BLE follow a separate test case matrix from products that support both modes. Most modern wireless IoT products use BLE only, which simplifies the qualification scope compared to dual-mode devices.
A Certification Declaration is a cryptographically signed blob issued by the Connectivity Standards Alliance following successful Matter conformance testing through a CSA Authorized Test Provider. The CD is embedded in the device’s firmware and validated during the Matter commissioning process. Without a valid CD, a device cannot be commissioned into a Matter network. The CD records the Vendor ID and the Product IDs that passed certification, and it is linked to the product’s entry in the Distributed Compliance Ledger.
OTA testing services for wireless products should cover both the happy path and the full failure matrix: partial image transfers, power interruptions at each stage of the update sequence, corrupted or truncated images, failed integrity checks, and rollback behavior. The happy-path test confirms that a valid update applies successfully. The failure-path tests confirm that the device recovers gracefully (returning to functional previous firmware rather than bricking) under every failure condition that can realistically occur in the field. This failure-mode coverage is what EN 303 645 provision 5.3 implies and what distinguishes production-grade OTA testing from a basic smoke test.
Yes. OTA failure testing (including partial write simulation, power interruption during transfer, and corrupt image handling) can be instrumented through a combination of hardware test fixtures (to trigger power loss at defined points in the update sequence) and software test scripts (to corrupt or truncate update images before transmission). Integrating this into a CI pipeline requires HIL test bench infrastructure but is feasible for most wireless products. The payoff is continuous validation that the OTA path handles the full failure matrix across every firmware release, not just the one that was manually tested before a launch.
CI/CD regression testing for wireless firmware means running a defined set of protocol-layer test cases automatically on every commit or nightly build, not just compiling the firmware and running unit tests. For a BLE device, this means running a subset of PTS test cases against real hardware on each build and flagging failures before they are buried under subsequent commits. For a Matter device, it means running the CSA’s Python-based conformance scripts as part of the build gate. The goal is to catch specification regressions within hours of introduction, not weeks later during a manual pre-release PTS run.
BLE and Wi-Fi share the 2.4 GHz ISM band and, on multi-radio devices, can interfere with each other’s transmissions. Coexistence testing involves running BLE and Wi-Fi simultaneously under traffic load conditions and measuring the impact on each protocol: connection establishment rate, packet error rate, throughput, and connection event timing. Devices with hardware coexistence signaling between the BLE and Wi-Fi radios require testing that validates the coexistence arbitration behavior specifically, not just the radio co-location under light load.
Further reading
- Automating Bluetooth Qualification: From Weeks of PTS to an Overnight Job: the engineering companion to Gap 3, covering how to replace a two-week manual regression with an automated overnight run.
- OTA Firmware Update Testing: The Failure Modes Nobody Tests Until a Device Bricks in the Field: deep engineering companion to Gap 5, the OTA path not validated under failure conditions.
- Bluetooth Low Energy Power Optimization: How to Get Weeks From a Battery That Lasts Days: the firmware-side engineering context for Gap 6, power budget measured once, not continuously.
- BLE Privacy by Design: MAC Randomization and EU Compliance: the BLE-specific regulatory and privacy side of Gap 7, covering MAC randomization, EU data protection alignment, and what EN 303 645 means for BLE-connected products.
