Smart Ignore Regions in Vizzly

We’ve been working on smart ignore regions in Vizzly. The short version: Vizzly can now find areas that keep changing across builds, ask you to confirm whether that change is expected, and use that decision on future reviews without turning the whole area into a blind spot.

This came out of a normal visual testing problem. Our own main build had a pile of screenshots waiting for review, and a lot of the noise was the same kind of thing you see in real apps all the time: footer text, timestamps, notification rows, seeded data, small bits of content that change between runs. You still want visual coverage on those pages. You just do not want to spend every review approving the same dynamic area again.

Most tools solve this with ignore regions. You draw a box, save it somewhere, and that part of the screenshot stops mattering to the comparison. I was against adding ignore regions to Honeydiff or Vizzly on principle for a long time because the tradeoff always felt wrong. Some tools push you toward masking the area with CSS or HTML during the test run, which is even worse to me. Now the test is looking at a version of the page that users never see.

Fixed boxes miss too much

Traditional ignore regions are fixed boxes. You draw a rectangle around the thing that changes and tell the visual testing tool to ignore whatever happens inside it.

That handles the obvious timestamp case. But what happens when the component shifts down inside that box? What happens when text wraps, overlaps another element, gets clipped, or renders in the wrong place? In a lot of tools, the answer is basically nothing. The region is ignored, so the tool does what you asked.

CSS and HTML masking have a similar problem from a different angle. You can hide the dynamic content before the screenshot is taken, but then you are not actually testing that UI anymore. You are testing a sanitized version of it. The real app still renders the timestamp, footer, avatar, live count, notification row, or whatever else your users are going to see.

There is also the setup problem. Someone has to go through the screenshots and decide which boxes to ignore or which pieces of the DOM to mask. Then the UI changes. New dynamic areas appear. Old ones move. Pretty soon you are maintaining a second set of visual testing rules made out of coordinates and test-only overrides.

We wanted to avoid that tradeoff. It is annoying to approve the same footer, timestamp, or generated row forever. But the fix should not be “stop checking this part of the UI” or “change the page so the visual test passes.” We want you to use real data and still test the UI users actually get.

That is not theoretical for us. I have purposefully avoided stabilizing a bunch of the data in Vizzly’s own visual test suite because I want our tests to exercise the real product. If the footer text, notification copy, or seeded content changes, Vizzly should understand that pattern instead of forcing us to either approve it forever or remove it from the page under test.

Honeydiff made this possible

Honeydiff gives us the data we need here. It does not just tell Vizzly that two screenshots are different. It gives us structured information about where the image changed: regions, line clusters, bounds, density, shape, and enough metadata to understand whether the same kind of change keeps showing up in the same place.

With that metadata, Vizzly does not need to start with a hand-drawn box. If the same footer line changes across a bunch of builds, we can group those observations into a candidate region. If the same notification area keeps changing in the same shape, we can surface that too.

This is one of the reasons I care so much about Honeydiff being part of Vizzly instead of a generic image diff bolted on at the edge. The comparison engine can produce the region metadata, Vizzly can store the history, and the review UI can turn that into a decision you make while looking at the screenshot. I wrote more about the foundation of that engine in the Honeydiff post.

You confirm the pattern

If you have spent any time in Vizzly, you have probably seen the footer change. We rotate a little line down there, and it kept showing up in review. So the footer became the test case for this feature.

When Vizzly sees that kind of repeated change, it shows the candidate region in the review UI. You can accept it as dynamic or reject it while looking at the screenshot. Small interaction, but that is the point. The decision happens where you already are, with the actual UI in front of you.

Once you accept it, Vizzly saves that decision for the project. Future builds can match new candidate regions against the confirmed region and use that during auto-approval.

The important detail is that Vizzly still checks the rest of the screenshot. A confirmed dynamic region is not an invisibility box. A comparison only gets auto-approved when the meaningful diff is covered by confirmed dynamic regions or other known-safe patterns. If there are changed pixels outside those areas, the screenshot stays pending.

The decision is narrower than a traditional ignore region. You are confirming that a recurring area is expected to change. You are not giving every future diff near that area a free pass.

What happened in our own build

We tested this on Vizzly’s own main build because dogfooding makes the tradeoffs obvious very quickly. Before confirming the big footer region, the build had 95 changed comparisons still pending review. After confirming that footer as dynamic content and rerunning the same main build, the next build dropped to 54 pending changed comparisons.

That is a pretty cheap win. One confirmation took 41 changed screenshots out of the review queue without changing the test suite or sanding down the UI just for the test.

The more important part is what did not happen. Some screenshots still stayed pending because the confirmed regions only covered part of their diff. Vizzly recognized the known dynamic area, but it did not approve the whole screenshot just because one region matched. The rest of the change still needed review.

The goal is to cut the repeated review work without turning a noisy region into bad approvals. I have shipped visual bugs this way. Whoops.

What comes next

There is still plenty to improve in the UI and in Honeydiff: clearer labels, better region matching, and better explanations for why a candidate is safe or risky. But the core idea is working. Vizzly can learn dynamic regions from real visual history, ask you to confirm the pattern in context, and reduce future review noise without hiding the screenshot from analysis.