ENG-000deployed

RAN performance test automation

Turning a manual, error-prone KPI validation process into a parallelized automation suite for live radio access network infrastructure.

Client

Altiostar, a Rakuten Symphony company — RAN engineering organization

Role

Senior Member of Technical Staff — owner of the automation suite end to end

Timeframe

Oct 2021 – present

Stack

Python · Selenium · Jenkins · Git · Multithreading · Linux · TM500

01 — Situation

Radio access network software releases have to be validated against performance KPIs on physical test setups before they ship. The existing process was heavily manual: engineers configured TM500 test cases by hand, watched throughput live, then collected crash files, logs, and KPI statistics after each run and compared them against expected values.

Manual validation didn't scale across the growing number of RAN setup flavours, and post-processing wait time between test cases was dead time on expensive lab hardware.

02 — Constraints

!Live network test infrastructure — a bad automation run can block a shared lab setup for an entire team.
!Multiple heterogeneous remote machine environments that the suite had to run on without per-machine tweaks.
!End users were test engineers, not developers — the suite had to be operable by people who would never read its source.
!KPI reports lived behind an EMS web interface with no API, so collection had to be automated through the browser.

03 — Stakeholders

Test engineers (end users)

Configuring runs easily and trusting the pass/fail verdicts

RAN development teams

Fast, reliable signal on whether their build regressed performance

Lab infrastructure owners

Maximizing utilization of expensive shared test hardware

04 — Architecture

01Execution engine drives TM500 test cases configured by the end user and prints live throughput and stats in parallel.
02Validation is decoupled from execution: run data is persisted immediately, so a suite failure never loses completed test-case results, and validation of run N happens while run N+1 executes.
03Log, crash-file, and KPI collection run on multiple threads in parallel to cut dead time between test cases.
04Selenium automates KPI report downloads from the EMS web interface.
05Jenkins pipelines chain build upgrade → test execution → consolidated reporting across multiple RAN setups; results consolidate into a single report on a log server.

05 — What shipped

✓A production automation suite used across all RAN setup flavours for performance validation.
✓CI/CD integration so that build upgrades automatically trigger test execution.
✓A consolidated reporting layer that replaced scattered per-run artifacts.

06 — Outcomes

70%

reduction in manual validation effort

50%

reduction in total suite execution time

All

RAN setup flavours covered by one suite

07 — Retro: what I'd do differently

Decoupling execution from validation was the single highest-leverage decision — I'd reach for it earlier next time instead of optimizing individual test cases first.

I underestimated how much of 'automation' work is actually stakeholder work: the suite only got adopted once pulling latest changes was a one-command operation for end users.

Selenium against an EMS UI is fragile by nature; with today's tooling I'd wrap that layer with retries and screenshot-on-failure from day one.

Questions about this engagement?

Ask the copilot →