CTG Enhances Test Management and Automation for Academic Medical Center

The Challenge and Opportunity

Our client is an academic medical center with multiple hospitals, clinics, and facilities in the Southeastern U.S. The client manages a diverse and complex software portfolio, including web applications, mobile apps, and Epic-integrated systems. As new applications were introduced and existing ones updated, they faced several testing challenges:

• Inconsistent test management practices across teams.
• Limited automation in testing, relying heavily on manual quality assurance (QA) and unit testing.
• Lack of standardized tools and processes for integration and system-level testing.
• Need for improved efficiency, traceability, and coverage in testing workflows.

The client sought a strategic partner to assess its current testing landscape—including workflows, tools, and team—and recommend a sustainable automation strategy to improve performance, compliance, and reliability. Based on our existing relationship and specialized expertise in healthcare testing, the client engaged CTG for help.  

The Solution and Results

To help our client evaluate automation technology, we partnered on a Test Management Assessment and Automation Pilot. Our testing experts began by gaining a comprehensive understanding of the client’s current environment by:

• Conducting interviews across multiple application teams.
• Taking inventory of applications, test plans, tools, and stakeholder roles.
• Identifying cultural and technical barriers to automation adoption.
• Evaluating functional, integration, performance, and security testing processes.
• Assessing tool effectiveness, continuous integration/continuous delivery (CI/CD) integration, and team readiness.
• Identifying gaps in defect management, test coverage, and documentation.

Based on this thorough evaluation, we delivered a roadmap for test management optimization that included recommendations for automation strategies tailored to our client’s application portfolio and toolsets based on viability, ease of use, and cost.

To help our client understand how automation technology would work in their environment, they selected two critical applications for automation pilots: one that ensures timely alerts for critical lab and radiology results and the other a web-based clinical educational app. Over a four-week period, our testing experts performed automation pilots on specific application workflows that compared open-source tools, Playwright (with Cucumber and JMeter), and a licensed tool, TestRigor, an AI-enabled low code automation tool. To accelerate testing, we used a custom Playwright framework that enabled developers to leverage pre-set features and common code.

The pilots demonstrated significant improvements in speed, scalability, and effort reduction: 

Alerts Application

• Automated workflows reduced effort by up to 85.4% compared to manual testing.
• Parallel execution further enhanced speed and scalability.
• Enabled Behavior-Driven Development using Cucumber and Playwright. 

Clinical Education Application

• Achieved 54.5% effort savings without wait times.
• Leveraged TestRigor’s AI-driven low-code interface for rapid script development.
• Improved dashboard verification and user enrollment testing.

CTG presented these final results to the client, including documentation, demo recordings, and tool evaluation matrices. We provided a weighted comparison of tools based on the client’s specific needs, highlighting pros and cons of each tool related to infrastructure, ease of use, and cost. The weighted score calculation sheet was shared to empower customers to conduct similar analyses using any automation tools in the future. Ultimately, we recommended a hybrid approach using both open-source and licensed tools.

The engagement has enabled our client to:

• Develop a comprehensive understanding of automation tools, including their features and relevance to specific application needs.
• Identify and adopt best practices and effective test strategies for implementing test automation.
• Strategically plan phased automation rollouts across additional applications to maximize efficiency and scalability.

Our client now has a clear roadmap and toolset recommendations to scale automation, improve quality assurance, and support its evolving digital ecosystem.