Setting Up Visual Regression Testing for Your Open Source Project

If you’re building an open source project with any kind of UI (web app, component library, documentation site, whatever), visual regression testing gives contributors and coding agents a shared baseline before they touch the interface. The problem is most visual tools either have restrictive free tiers or cost more than volunteer maintainers want to spend.

My career has really been helped a ton by open source, so I want to give back what I can. Hopefully this is generous enough. Let me show you how to get it set up.

What You Actually Get

First, let’s talk about what’s included. Qualifying open source work gets a public-project Team review workflow for free, within Open Source limits:

• 75GB storage
• Up to 10 team members
• 6 concurrent build workers
• 90-day build retention
• Position-based comments for reviewing changes
• GitHub integration with commit status updates
• Identical comparisons need no review
• Build and screenshot review for public projects
• Git-based baseline management

Your Vizzly projects need to stay public on this plan. Vizzly can use your linked GitHub repository to check eligibility. A mention of Vizzly in your project is always appreciated, but it is not required.

Getting Approved

The signup process is refreshingly simple. No application essays or waiting for manual approval.

Start by creating a Vizzly account if you don’t have one. Then:

1. Make your organization public - In your organization settings, toggle “Public Organization” on
2. Set projects to public - Any projects you create need public visibility
3. Apply for the open source plan - You’ll see an “Apply for Open Source Plan” button in billing settings
4. Get instant approval - If you meet the requirements, you’re approved immediately

You can get approved before connecting GitHub, but linking GitHub helps Vizzly keep eligibility checks and PR context working.

Getting Started

Once you’re approved, install the CLI and add screenshot capture to your tests with a Vizzly integration or vizzlyScreenshot(). Then run those tests with Vizzly:

# Install the CLI (requires Node.js 22+)
npm install -D @vizzly-testing/cli

# Set your API token from your Vizzly project settings
export VIZZLY_TOKEN=your-api-token

# Run your tests with Vizzly
npx vizzly run "npm test" --wait

For a local check, run the same screenshot tests in TDD mode:

npx vizzly tdd run "npm test" --no-open

Check out the full getting started guide for more setup options and integrations.

GitHub Integration

GitHub integration is still the easiest way to get PR checks and eligibility context in place. Link the repository that owns the UI work so Vizzly can attach reviewed visual context to pull requests.

Connect Your Repository

1. Go to your project settings in Vizzly
2. Click “Connect GitHub Repository”
3. Select the repository that owns the UI work
4. Install the GitHub App if prompted

Check out the GitHub integration guide for detailed setup instructions and the CI/CD docs for adding Vizzly to your workflows.

Real-World Example

Let’s say you’re maintaining a component library. A contributor submits a PR that changes your button component. Here’s what happens:

1. Local development - The contributor uses vizzly tdd to see their changes in real-time
2. PR created - GitHub Actions automatically runs visual tests
3. Review process - You can see exactly what changed visually and leave position-based comments
4. Approval - Once you approve the visual changes, the PR can be merged

The whole team can collaborate on the visual review, and you have an audit trail of all visual changes.

Works with Any Screenshot Source

Unlike other visual testing tools that force you into their rendering pipeline, Vizzly works with any screenshot source. Building a documentation site with Gatsby? A mobile app with React Native? A design system with Storybook? Screenshots from BrowserStack or Sauce Labs?

As long as you can capture screenshots (and you’re building in the open), you can use our visual testing workflow. No vendor lock-in, no forced rendering environments.

Give It a Try

If you’re maintaining an open source project that could benefit from visual testing, sign up for Vizzly and apply for the open source plan.

The eligibility check is automated, so you’ll know right away if you qualify. And if you run into any issues, feel free to reach out - we’re pretty responsive to the open source community.

Visual bugs are just as frustrating in open source projects as they are in commercial ones. Might as well catch them before your users do.