QA Automation Tool
Run tests on Studio Preview URLs. Describe what exists—the system validates and reports.
Describe → Run Test → View results
How It Works
1
Describe
Enter test name, type, and preview links. Add expected clickthrough (and QA URL for High Impact).
→
2
Run Test
The system captures screenshots and validates against your test type rules.
→
3
View Results
Review run summary and per-unit reports: pass/fail, blockers, warnings, screenshots. Open past runs from Reports.
Run Test
Describe what exists. The system validates and reports. All fields below are required unless marked optional.
📋
Testing:
💡
What's happening now:
Ready to start test...
Screenshots
Screenshots
Comprehensive Test Results
Test History (Last 30 Days)
Recent runs from the last 30 days. For search and full history, use .
Loading...
Past Reports
Find, open, and review past QA test runs. Reports are read-only.
| Test Name | Run Date & Time | Test Type | Units | Result |
|---|
Loading reports…
Validation Guidelines Configuration
Optional admin configuration for validation rules (dimensions, ad types, brands, global defaults). On Run Test, the system applies rules based on your Test Type selection; you do not configure features there.
Brand Management
Create and manage brands with custom validation rules. Each brand can have its own file size limits, required elements, and dimension-specific rules.
Dimension-Specific Guidelines
Set file size limits, animation duration, and required elements for each ad dimension.
DM
No dimensions configured
Click "Add Dimension" to get started
Ad Type Guidelines
Configure guidelines for different ad formats (Animated Rich Media, Video, Responsive, etc.).
AT
No ad types configured
Ad types are managed automatically based on detected formats
Global Default Guidelines
Set default values that apply to all ads unless overridden by dimension or ad type settings.
Create Brand
Duplicate Brand
Analytics Dashboard
View test statistics, trends, and brand performance metrics
TS
0
Total Tests
SR
0%
Success Rate
TM
0s
Avg Duration
BR
0
Brands Tested
Test Results Trend
Error Categories
Brand Performance Comparison
Recent Activity
QA Testing Checklist
What the system checks. Validation is driven by your Run Test selection (Test Type); this list is for reference only.
Base Functionality
✓ Entire ad is clickable
✓ URL matches expected destination
✓ File size within limits (4MB standard, 250KB TWC)
Design Requirements
✓ All assets are high quality and not pixelated
✓ Proper padding and layout (no cropped elements)
✓ All visible text is spelled correctly
✓ Brand colors and fonts match guidelines
Animation Standards
✓ Main visual enters smoothly within 3-5 seconds
✓ Transitions are seamless with no abrupt cuts
✓ Animation duration meets requirements (< 30s standard, < 15s TWC)
✓ End frame displays key information (CTA, URL)
Responsive Design
✓ Layout adapts correctly at all breakpoints
✓ Text remains readable at all sizes
✓ Interactive elements work on mobile devices
Video Requirements
✓ Video end card is present
✓ Autoplay settings are correct (muted, no audio by default)
✓ Video is not letterboxed
✓ Timeline matches video duration
Tracking & Metrics
✓ Tracking scripts are properly implemented
✓ Engagement metrics events are configured
✓ Click-through tracking works correctly
Help & Definitions
Welcome to the QA Automation Tool help center. Here you'll find definitions, explanations, and guidance for using the tool effectively.
Key Terms & Definitions
Preview Link (Studio Preview URL)
A hosted creative preview link from Studio/Mixpo. Format:
https://studio.mixpo.com/videoad/[ID]/[ad-name].
Enter one URL per line in Preview Links. For Rich Media you can add multiple links; for High Impact exactly one. The tool loads each link and runs validation for the selected Test Type.
Google Drive Folder
The URL of the Drive folder that contains (or will contain) assets for the creative. Used for file presence and asset validation. Structure is validated after the test runs.
High Impact (Skyclimber, Pinnacle, Canvas Scroller)
Responsive ad formats that adapt to screen size. When you select a High Impact Test Type, you must provide exactly one Preview Link and a QA URL (used to execute High Impact validation; distinct from the preview link).
Breakpoint
Specific screen widths where responsive ads change their layout. The tool captures screenshots at standard breakpoints (320px, 768px, 1024px, 1280px, 1920px)
to verify the ad displays correctly across all device sizes.
ClickTag
The click-through URL that redirects users when they click on the ad. The tool validates that the entire ad is clickable and that the ClickTag URL
is properly configured and accessible.
Asset Health
Validation of all files (images, videos, JavaScript, CSS, fonts) used by the creative. The tool checks for missing assets, broken links,
invalid file formats, and network errors. Results show total assets, valid assets, and any broken or missing files.
Interactive Elements
User-triggered components like polls, galleries, scratch-off reveals, countdown timers, and videos. The tool automatically detects and tests these elements
to ensure they function correctly when users interact with them.
Bedrock Vision Analysis
AI-powered visual analysis using AWS Bedrock (Nova) that examines screenshots to detect logos, CTAs, URLs, text content, and visual issues.
Provides contextual understanding of the ad's visual elements and identifies potential problems like cropping or pixelation.
Vision API Analysis
Google Cloud Vision API analysis providing precise OCR (text extraction), object detection, logo recognition, and color analysis.
Works together with Bedrock for comprehensive visual understanding.
Beginning Frame
The first frame visible when the ad loads. Analyzed for initial visual state, animation start, and overall quality.
Used to verify the ad loads correctly and animations begin as expected.
End Frame
The final frame after animations complete. Validated to ensure it contains required elements: call-to-action (CTA) and URL.
Critical for ensuring users see necessary information.
Ad Format Detection
Automatic identification of the ad type (Animated Rich Media, Video Rich Media/IBV, Skyclimber, Pinnacle, Canvas Scroller) based on URL patterns,
content analysis, and file structure. Ensures appropriate tests are run for each format.
Validation Guidelines
Configurable rules for file size limits, animation duration, required elements, and quality standards. Set dimension-specific or ad-type-specific guidelines
in the "Configure Guidelines" section. Tests are validated against these rules.
📋 Test Categories Explained
Functional Tests
Validates core functionality: clickability, URL correctness, file size compliance, and basic ad behavior.
Asset Health
Checks all files load correctly, validates file formats, detects broken links, and monitors network requests.
Responsive Testing
Captures screenshots at multiple breakpoints to verify the ad adapts correctly to different screen sizes.
Animation Analysis
Validates animation timing, duration, transitions, and ensures the end frame displays all required elements.
Interactive Elements
Tests polls, galleries, scratch-off, timers, videos, and expand/collapse functionality.
Visual Analysis
AI-powered detection of CTAs, URLs, text content, and visual quality issues.
Data Extraction
Comprehensive extraction of all visible text, images, videos, links, fonts, colors, and tracking scripts from the ad.
Error Detection
Monitors JavaScript errors, console warnings, network failures, and other runtime issues.
Quick Start Guide
Step 1: Fill Run Test Fields
Test Name: A label for the run (e.g. Campaign A - 300x250).
Test Type: Rich Media (RM) or one of High Impact — Skyclimber, Canvas Scroller, Pinnacle. This drives which validations run.
Google Drive Folder: URL of the Drive folder for asset validation.
Preview Links: One Studio Preview URL per line. RM allows multiple; High Impact allows exactly one.
Expected Clickthrough URL(s): One per line to match each Preview Link, or a single URL to apply to all.
For High Impact, also fill QA URL. For RM, optionally check TWC Placement.
Test Type: Rich Media (RM) or one of High Impact — Skyclimber, Canvas Scroller, Pinnacle. This drives which validations run.
Google Drive Folder: URL of the Drive folder for asset validation.
Preview Links: One Studio Preview URL per line. RM allows multiple; High Impact allows exactly one.
Expected Clickthrough URL(s): One per line to match each Preview Link, or a single URL to apply to all.
For High Impact, also fill QA URL. For RM, optionally check TWC Placement.
Step 2: Run Test
Click "Run Test" when all required fields are valid. The form locks and status shows "Running validation on X unit(s)…".
The system loads each preview link, captures screenshots, and validates against the Test Type rules. Typically 30–60 seconds per unit.
Step 3: Review Results
View the run summary (pass/fail per unit, blockers, warnings) and open individual unit reports. Use Reports to find and reopen any past run.
❓ Frequently Asked Questions
Q: What file size limits apply?
Validation rules (including file size where applicable) depend on Test Type and TWC placement. Standard and TWC limits (e.g. 4MB standard, 250KB TWC) can be viewed or adjusted in Guidelines configuration if available to your role.
Q: Do I need API keys?
Bedrock Credentials: Required for AI-powered visual analysis. Configure AWS access key ID, secret access key, region, and model ID in environment variables.
Vision API Key: Optional for enhanced OCR and object detection. Get it from Google Cloud Console.
The tool works without these keys but won't include AI analysis features.
Vision API Key: Optional for enhanced OCR and object detection. Get it from Google Cloud Console.
The tool works without these keys but won't include AI analysis features.
Q: What ad formats are supported?
All standard HTML5 ad formats: Animated Rich Media, Video Rich Media (IBV), Responsive Skyclimber, Responsive Pinnacle, and Responsive Canvas Scroller.
The tool automatically detects the format and runs appropriate tests.
Q: Can I run multiple units in one test?
Yes for Rich Media: add multiple preview links (one per line). Each link is one unit and gets its own report. High Impact runs support exactly one preview link per run.
Q: What is the QA URL (High Impact only)?
The QA URL is used to execute High Impact validation and is different from the preview link. Both are required when Test Type is Skyclimber, Canvas Scroller, or Pinnacle.
Q: How long do tests take?
Typical test duration: 30-60 seconds. This includes file extraction, browser automation, screenshot capture, asset analysis, AI processing, and report generation.
Progress is shown in real-time.
Q: What are the test statuses?
PASS: Test completed successfully, all criteria met.
FAIL: Test failed, critical issue detected.
WARNING: Test passed but with potential issues or recommendations.
NO DATA: Test couldn't be performed (missing data or API keys).
NOT TESTED: Test not applicable or not yet implemented.
FAIL: Test failed, critical issue detected.
WARNING: Test passed but with potential issues or recommendations.
NO DATA: Test couldn't be performed (missing data or API keys).
NOT TESTED: Test not applicable or not yet implemented.
Troubleshooting
Server Won't Start
Kill existing processes:
pkill -f "tsx.*server" then run npm run ui again.
Ensure port 3000 is available or set PORT environment variable.
Tests Not Running
Check browser console for errors. Verify Playwright browsers are installed:
npx playwright install.
Ensure API keys are configured in .env file if using AI features.
Run Test Button Stays Disabled
All required fields must be valid: Test Name, Test Type, at least one Preview Link (exactly one for High Impact), at least one Expected Clickthrough URL, and QA URL when Test Type is High Impact. Check for inline errors or helper text.
Bedrock Analysis Not Working
Verify BEDROCK_ACCESS_KEY_ID, BEDROCK_SECRET_ACCESS_KEY, BEDROCK_REGION, and BEDROCK_MODEL_ID are set in
.env file. Check credentials are valid and have proper permissions.
Review server logs for API errors. The tool works without Bedrock but won't show AI analysis.