Back to blog
Secure SaaS Deployment9 min read

When the Fix Isn't Upstream: Should You Pause, Patch, or Ship?

A practical framework for deciding whether to pause a release, accept a temporary risk, or carry a manual patch when an upstream dependency fix is not ready.

Dependency SecuritySoftware Supply ChainSecure DeploymentRisk Management
When the Fix Isn't Upstream: Should You Pause, Patch, or Ship? article preview image

A security alert can become a release decision

Recently, I ran into a high-severity vulnerability in a transitive dependency used by my AWS infrastructure tooling. The vulnerable package was not something I had selected directly. It arrived through another dependency, and the upstream package had not yet published a release that gave me a clean, supported upgrade path.

I also had a patch that needed to ship that day.

That turned an automated security finding into a real operational decision. I could hold the deployment and wait for upstream maintainers, or I could carry the dependency update myself, validate it, and accept responsibility for that temporary fix.

I did not particularly want the second option. A manual dependency fix creates work that the normal package update path is supposed to handle. But waiting was not neutral either. It would have delayed a needed patch for an unknown amount of time.

I chose to make the update and deploy. The interesting part was not the command that changed the dependency tree. It was deciding which risk I was prepared to own.

High severity is a signal, not the whole decision

A severity rating is useful because it tells a team where to look first. It does not, by itself, describe the risk to a particular system.

The same vulnerable package can appear in very different places. It might process public input inside a production API. It might run only during a local test. It might be part of a build tool that operates on repository-controlled files. Those situations should not receive identical responses simply because the scanner prints the same severity label.

Before deciding what to do, I needed to answer more specific questions:

  • Is the vulnerable code present in the deployed application, or only in build and infrastructure tooling?
  • Can untrusted input reach the affected behavior?
  • What credentials, network access, or deployment authority does the process have?
  • What would successful exploitation actually do in this environment?
  • Is there a known patched version of the transitive package, even if the parent dependency has not adopted it yet?

In this case, the package was part of a CDK and build dependency chain rather than the public application runtime. That reduced the practical exposure because normal site visitors could not send it input directly. It did not make the finding irrelevant. Build and deployment tools execute code, read the repository, and can run with credentials capable of changing infrastructure.

The right conclusion was not "high means stop everything" or "build-only means ignore it." The right conclusion depended on reachability, authority, impact, and the available mitigations.

Waiting is also a security decision

Pausing a release can be the correct response. If vulnerable code is reachable from untrusted input, the impact is serious, and no safe mitigation exists, shipping may create more risk than waiting.

But holding a deployment has consequences too. The blocked release may contain a security patch, an operational fix, or a reliability improvement. The upstream maintainer may not provide a timeline. A short delay can become an indefinite one while other changes accumulate behind it.

That matters because "do nothing" is still a decision. It means keeping the current dependency state and postponing every change tied to that release path. The risk of waiting needs to be compared with the risk of carrying a temporary fix, not treated as if it were risk-free.

For me, the release pressure was legitimate. The change needed to go out, the vulnerable path was not exposed to public input, and a narrow dependency update could be tested. Waiting for an unknown upstream date was the less controllable option.

The available options were all imperfect

When an upstream fix is not ready, a team usually has four broad choices.

Pause the release

This avoids introducing a knowingly vulnerable version into a new build, but it also blocks every fix behind that release. It is strongest when exploitability is high and the delay is likely to be short.

Accept the finding temporarily

A documented exception can be reasonable when the vulnerable behavior is unreachable and compensating controls are strong. This should have an owner and an expiration date. "Temporary" should not quietly become permanent.

Replace the parent dependency

Moving to another library can remove the vulnerable chain, but a replacement made under time pressure may introduce a much larger change. For foundational tooling such as AWS CDK, replacement is rarely a same-day decision.

Carry the update yourself

An override, lockfile update, patch, or small fork can move the transitive package to a corrected version before the parent package does. This can be the narrowest practical option, but it transfers compatibility testing and maintenance responsibility to your team.

None of these choices is automatically correct. The decision should minimize total risk, not merely make the scanner quiet.

Why I carried the dependency update

I chose the manual path because several facts lined up:

  • The dependency was used by controlled build and infrastructure code, not by a public request handler.
  • The vulnerable behavior was not receiving attacker-controlled patterns in the application.
  • The dependency change was narrow and visible in the lockfile.
  • The surrounding CDK application had tests and synthesis checks that could detect compatibility problems.
  • The release being held had a real reason to ship.
  • There was no reliable date for an upstream release that would resolve the decision for me.

I updated the dependency chain, reviewed what moved, adjusted the build runtime required by the newer CDK tooling, and ran the relevant tests and production build. The goal was not to claim that a package update made the system permanently safe. The goal was to make a bounded change, prove that the infrastructure still synthesized correctly, and avoid knowingly carrying the original vulnerable resolution into the deployment.

That distinction matters. A passing audit is evidence about the dependency graph and advisory data at that moment. It is not a lifetime guarantee. Advisories can be revised, incomplete fixes can be discovered, and a previously acceptable version can be flagged again.

A manual patch changes ownership

Once a team moves ahead of upstream, the technical fix is only part of the work. The team now owns the gap between its local resolution and the parent package's supported dependency tree.

That ownership should be explicit:

  1. Record why the change was necessary and which advisory or failure it addresses.
  2. Keep the override or patch as small as possible.
  3. Lock the resulting dependency graph so CI and local builds use the same versions.
  4. Test the behavior that the parent dependency relies on, not only application code.
  5. Track the upstream package and remove the workaround when an official release is available.
  6. Re-run audits because advisory ranges and recommended versions can change.

The maintenance burden is the main reason I was reluctant to patch manually. It is easy to add an override. It is harder to remember why it exists six months later, determine whether it is still needed, and prove that removing it is safe.

The patch therefore needs an exit plan. Without one, a temporary security response becomes another opaque piece of infrastructure.

Build dependencies deserve serious review

It is tempting to dismiss a finding because the package does not ship in the browser or run inside a Lambda. Runtime reachability is important, but build dependencies have their own trust boundary.

A CI process can read source code, install packages, create deployment artifacts, assume AWS roles, publish assets, and update infrastructure. Code executing there may have more authority than ordinary application code, even when no customer can call it directly.

That is why the CDK detail mattered in this case. AWS CDK was not the subject of the article because it is uniquely risky. It was relevant because infrastructure tooling sits close to deployment credentials and production change control. A transitive vulnerability in that path deserves a thoughtful response, even when its immediate exploitability is limited.

The same reasoning applies to bundlers, test runners, image processors, code generators, deployment CLIs, and package-manager hooks.

A practical decision record

When this happens again, I want the decision to be reproducible rather than based on the stress of the moment. A short written record should answer:

  • What package and advisory triggered the decision?
  • Is it a direct or transitive dependency?
  • Where does it execute?
  • Can untrusted data reach the vulnerable behavior?
  • What permissions does that process have?
  • What release is being blocked, and what is the cost of waiting?
  • Which alternatives were considered?
  • What tests prove the workaround is compatible?
  • Who owns the workaround, and when will it be reviewed?
  • What upstream event allows the workaround to be removed?

This does not need to become a long governance exercise. A concise decision record is enough to replace vague memory with evidence.

It also helps during review. Instead of arguing about whether a scanner is too strict or not strict enough, the team can discuss the actual execution path and the risks attached to each option.

The least comfortable option can be the most controllable

I would still prefer an official upstream fix. It preserves the supported dependency path, reduces local maintenance, and lets the package owner carry compatibility responsibility.

But secure engineering does not always offer a clean option on the day a release needs to move. Sometimes the choice is between waiting without a timeline and carrying a carefully tested change yourself.

In this case, I chose the change I could inspect, test, document, and reverse. That did not eliminate risk. It made the risk explicit and gave it an owner.

That is the broader lesson for SaaS teams. Dependency security is not only about updating packages quickly. It is about understanding where code runs, deciding which risk is acceptable, and being honest about the maintenance responsibility created by the decision.

This is also why security belongs in the delivery path from the beginning. Automated audits, reproducible lockfiles, infrastructure tests, and controlled promotion do not make hard choices disappear. They provide enough evidence to make those choices deliberately. For more on that foundation, see The Problem with Bolting Security Onto SaaS Later.

GagliTech helps teams build secure AWS delivery systems and practical software supply-chain controls. For secure cloud deployment support, contact GagliTech.