Soft-yanking

[pirj]

Problem

Occasionally, when maintainers publish new gem versions, they make mistakes.

Examples:

1. rails Ruby version constraint mistake that broke 5.2.4.3-5.2.4.5 on Ruby 2.2. See https://github.com/rails/rails/blob/v5.2.4.3/activesupport/lib/active_support/cache/redis_cache_store.rb#L323
   It has been fixed in May 2020, but only released nearly a year later in Rails 5.2.4.6 (May 2021).

2. rspec-rails Ruby version constraint mistake that broke rspec-rails on Ruby 2.2.

3. diff-lcs issue with older Ruby versions

4. cucumber 4.0.0 broke compatibility due to diff-lcs dependency, fixed in 4.0.1 by pinning diff-lcs version to ~> 1.3.

Suggestion: Soft-yank

What soft-yanking means?

Gem maintainer scenario

The maintainer can soft-yank a gem version, just like they can yank it:

gem soft-yank GEM -v VERSION [-p PLATFORM] [--key KEY_NAME] [--host HOST]

Server/CI scenario

It remains possible to install the soft-yanked version of a gem with bundle install from Gemfile.lock.
Bundler emits a warning.

Developer scenario

Bundler excludes soft-yanked versions from dependency resolution.
bundle update/bundle lock show an error, just like for a yanked gem version or a removed gem.

Could things have gone better?

rspec-rails 4.0.0 could have been soft-yanked.
cucumber 4.0.0 could have been soft-yanked.
diff-lcs 1.4.3 could have been soft-yanked.

I have no such certainty regarding Rails, since it took a year to release the fix.

Misc

Related: rubygems/rubygems#1506 (comment)

#26 is semi-related, a proposal to prevent the only cause I'm practically aware of, weak Ruby version constraint. There might be others, like adding extra runtime dependencies, but I have not seen this in the wild.

cc @halostatue @JonRowe @marcandre @mattwynne @aslakhellesoy.

[simi]

🤔 It seems majority of examples are related to old EOL Ruby. I suggest to upgrade instead looking for bandaid.

[marcandre]

Sounds to me like a good idea. It's not clear to me what the advantages of "hard-yanking" over this.

[pirj]

suggest to upgrade

For the user standpoint? Practically, it's often easier said than done. See e.g. GitLab 2.5.3 -> 2.6.3.

From maintainers' standpoint?
rspec supports Ruby down to 1.8.7 until now in version 3.x. For ten years after Ruby 1.8.7 EOL.
rspec will keep supporting Ruby 2.3 that is EOL for 2.5 years now in the upcoming 4.0, for probably another five years until RSpec 5.0.
rails 5.2.0 was released on April 09, 2018 supported Ruby >= 2.2.2, and Ruby 2.2 EOL on 31 Mar 2018. The last of the 5.2.x to date, 5.2.6, was released May 05, 2021.
I believe they have their reasons to support users that stick with older Ruby versions.

[simi]

@pirj gem support isn't relevant at all. Related Ruby itself is not supported anymore and you should not use it.

🤔 If I remember well, yank of gem is recommended to be done only under some circumstances (for example when you include some secrets as part of the gem bundle by mistake). If you just release broken version, you can fix it by releasing another one.

Can you explain on your examples why releasing new version isn't enough to fix the problem and how soft-yanking would solve the problem instead?

[pirj]

the advantages of "hard-yanking" over this

Hard-yanking should stay for cases like

• mistakenly introduced severe security vulnerabilities or bugs that could irreversibly corrupt data [1]
• hijacking

IANAL, but probably not for licensing issues.

[pirj]

Can you explain on your examples why releasing new version isn't enough to fix the problem

Easy.

# Gemfile
gem 'rspec-rails'

Releasing rspec-rails 4.0.1 didn't help because bundle was still resolving 4.0.0 as a matching version for Ruby 2.2.

See the link from the description for details.

[simi]

Releasing rspec-rails 4.0.1 didn't help because bundle was still resolving 4.0.0 as a matching version for Ruby 2.2.

And and wouldn't be locking to minor version enough for you in this case?

# Gemfile
gem 'rspec-rails', '~> 3.9'

[pirj]

You got straight to the point.
One gem soft-yank rspec-rails -v 4.0.0 command issued by one gem maintainer vs thousands of projects having to specify version constraints in their Gemfiles.

[simi]

@pirj that's from your point of view. Another projects will get into trouble even they don't care in this case getting (and also potentially reporting) annoying warning. And since in your case the reason is EOL Ruby, I think maintainer should prefer comfort for officially supported Ruby versions and this this is not valid reason for any kind of yank (even soft) IMHO.

[pirj]

I apologize for making this point, but I'm speaking not just as a developer of a project who is too lazy to update Ruby version, but as one of RSpec maintainers.

[marcandre]

@pirj that's from your point of view. Another projects will get into trouble even they don't care in this case getting (and also potentially reporting) annoying warning.

My understanding is that the warning would only occur for anyone who happened to have locked on the 4.0.0 version. It would disappear as soon as they follow instructions, which would install the correct version.

Any other users would have the correct version installed (i.e. the latest 3.x.x or 4.0.1 depending on their set).

[simi]

@marcandre according to

Server/CI scenario
It remains possible to install the soft-yanked version of a gem with bundle install from Gemfile.lock.
Bundler emits a warning.

Developer scenario
Bundler excludes soft-yanked versions from dependency resolution.
bundle update/bundle lock show an error, just like for a yanked gem version or a removed gem.

I assumed anyone using soft-yanked version will get warning or error.

@pirj no need to apologise, I'm just sharing my ideas and trying to find out potential problems to discuss.

[pirj]

the advantages of "hard-yanking" over this

Speaking of hard-yanking, I stick to the belief this should be reserved exclusively to the scenario when this is coordinated with the RubyGems team.
Otherwise, someone in the bad mood can ruin the day of on-call engineers of many projects, who will spend their evenings trying to understand why their container clusters lost the ability to scale dynamically.

But it's an off-topic, better suited for rubygems/rubygems#1506 or another RFC.

[marcandre]

I assumed anyone using soft-yanked version will get warning or error.

Right. To use a soft-yanked version you must have done a gem install before it was soft-yanked and thus got it in your Gemfile.lock though. bundle update would resolve the warning too.

[simi]

You got straight to the point. One gem soft-yank rspec-rails -v 4.0.0 command issued by one gem maintainer vs thousands of projects having to specify version constraints in their Gemfiles.

🤔 Also getting back to this solution. Wouldn't be enough to release 4.0.0.1 version, same as the 4.0.0 just doing Ruby version check in runtime and raising friendly error to lock to ~> 3.9 for old rubies? That way this problem can be fixed by bundle update and following printed instructions (non-locked bundle install will work by default).

[marcandre]

You got straight to the point. One gem soft-yank rspec-rails -v 4.0.0 command issued by one gem maintainer vs thousands of projects having to specify version constraints in their Gemfiles.

🤔 Also getting back to this solution. Wouldn't be enough to release 4.0.0.1 version, same as the 4.0.0 just doing Ruby version check in runtime and raising friendly error to lock to ~> 3.9 for old rubies?

This assumes that this 4.0.0.1 gem can successfully be installed and run on older Ruby...

[simi]

You got straight to the point. One gem soft-yank rspec-rails -v 4.0.0 command issued by one gem maintainer vs thousands of projects having to specify version constraints in their Gemfiles.

thinking Also getting back to this solution. Wouldn't be enough to release 4.0.0.1 version, same as the 4.0.0 just doing Ruby version check in runtime and raising friendly error to lock to ~> 3.9 for old rubies?

This assumes that this 4.0.0.1 gem can successfully be installed and run on older Ruby...

This just assumes 4.0.0.1 gem can be successfully installed on older Ruby. It will immediately raise (and print explaining message) on older Ruby once required -> there is no need to make it really compatible.

[marcandre]

there is no need to make it really compatible

Understood, but depending on the issue (an unresolvable dependency, etc.) it can be difficult to do from the 4.x code.
It is true that in general a 4.0.0.1 can be created out of the 3.x branch, and modified to issue the warning, and a 4.0.0.2 (that fixes the 4.0.0 setup) can be released.
This is quite ugly, and the requirement to manually lock to "~3.9" is also ugly and also requires a comment that this can be relaxed when Ruby is bumped above some version.

[simi]

@pirj @marcandre regarding

Could things have gone better?
rspec-rails 4.0.0 could have been soft-yanked.
cucumber 4.0.0 could have been soft-yanked.
diff-lcs 1.4.3 could have been soft-yanked.

If I understand it well, those problems will solve picking non-supported old rubies thanks to missing ruby_version constraint. Anyway if I understand it well, even those versions will get soft-yanked, this will need update of RubyGems/Bundler client itself.

🤔 Current minimal Ruby supported is 2.3 and there is plan to drop it (rubygems/rubygems#3260). So in theory, even this will get shipped, old clients (RubyGems on Ruby 2.2) will not be able to use this feature and all reported example problems will be still present.

[simi]

there is no need to make it really compatible

Understood, but depending on the issue (an unresolvable dependency, etc.) it can be difficult to do from the 4.x code. It is true that in general a 4.0.0.1 can be created out of the 3.x branch, and modified to issue the warning, and a 4.0.0.2 (that fixes the 4.0.0 setup) can be released.

Releasing 4.0.0.1 based on 3.x branch doesn't make sense since it in theory can be still provided for clients not facing any problems (for example having gem locked to ~> 4.0.0). My idea was based on special release based on 4.0.0 just adding new line to entry point like https://github.com/rspec/rspec-rails/blob/d810df59bb3b4ea7f3a2a0d0b169f2f2b9d390ac/lib/rspec-rails.rb#L1.

raise "rspec-rails 4.x and newer is not supported on Ruby #{RUBY_VERSION}, please lock rspec-rails to ~> 3.9 in your Gemfile." if RUBY_VERSION < '2.2'

This is quite ugly, and the requirement to manually lock to "~3.9" is also ugly and also requires a comment that this can be relaxed when Ruby is bumped above some version.

IMHO There is nothing ugly on locking your gems if newer version is not supported yet. For example currently I do this for psych 4, since it is not compatible with our app yet.

And also that's the price you pay for using really old Ruby in this case. 🤷

[marcandre]

Releasing 4.0.0.1 based on 3.x branch doesn't make sense since it in theory can be still provided for clients not facing any problems (for example having gem locked to ~> 4.0.0). My idea was based on special release based on 4.0.0 just adding new line to entry point like https://github.com/rspec/rspec-rails/blob/d810df59bb3b4ea7f3a2a0d0b169f2f2b9d390ac/lib/rspec-rails.rb#L1.

I understand this, but it may not be easy/feasible. In your example, you assume:

1. that /lib/rspec-rails.rb is parsable by old Ruby
2. that the dependencies successfully install

Maybe the situation is better now, with the latest available bundler for Ruby 2.2, the gem rspec-rails 4.0.0 does not install successfully:

$ bundle
Fetching gem metadata from https://rubygems.org/...............
Fetching version metadata from https://rubygems.org/...
Fetching dependency metadata from https://rubygems.org/..
Resolving dependencies...
Using rake 10.1.0
Installing concurrent-ruby 1.1.9

Gem::InstallError: i18n requires Ruby version >= 2.3.0.
An error occurred while installing i18n (1.8.10), and Bundler cannot continue.
Make sure that `gem install i18n -v '1.8.10'` succeeds before bundling.

AFAIK, there's no way for spec-rails to print anything unless it removes i18n as an (indirect?) dependency, which I think is impossible for the 4.x feature set.

IMHO There is nothing ugly on locking your gems if newer version is not supported yet.

Absolutely, if this is due to your app's code. If it is not supported yet because of dependencies, then I have a different opinion and believe it is the job of the gems to accurately describe those dependencies, and that of the dependency manager (bundler) to handle those dependencies.

[simi]

Releasing 4.0.0.1 based on 3.x branch doesn't make sense since it in theory can be still provided for clients not facing any problems (for example having gem locked to ~> 4.0.0). My idea was based on special release based on 4.0.0 just adding new line to entry point like https://github.com/rspec/rspec-rails/blob/d810df59bb3b4ea7f3a2a0d0b169f2f2b9d390ac/lib/rspec-rails.rb#L1.

I understand this, but it may not be easy/feasible. In your example, you assume:

1. that /lib/rspec-rails.rb is parsable by old Ruby
2. that the dependencies successfully install

Maybe the situation is better now, with the latest available bundler for Ruby 2.2, the gem rspec-rails 4.0.0 does not install successfully:

$ bundle
Fetching gem metadata from https://rubygems.org/...............
Fetching version metadata from https://rubygems.org/...
Fetching dependency metadata from https://rubygems.org/..
Resolving dependencies...
Using rake 10.1.0
Installing concurrent-ruby 1.1.9

Gem::InstallError: i18n requires Ruby version >= 2.3.0.
An error occurred while installing i18n (1.8.10), and Bundler cannot continue.
Make sure that `gem install i18n -v '1.8.10'` succeeds before bundling.

AFAIK, there's no way for spec-rails to print anything unless it removes i18n as an (indirect?) dependency, which I think is impossible for the 4.x feature set.

OK, clear. It is not about rspec-rails itself, but also related to its dependencies. This scenario should be improved in newer Bundler/RubyGems if I understand it well and install only supported version.

IMHO this is also the price for using old Ruby. Ruby 2.2 EOL (of security maintenance) was 3 years ago. If all you need to do to fix this problem currently is to lock rspec-rails to ~> 3.9 it is still super simple solution (considering how old Ruby is used here).

IMHO There is nothing ugly on locking your gems if newer version is not supported yet.

Absolutely, if this is due to your app's code. If it is not supported yet because of dependencies, then I have a different opinion and believe it is the job of the gems to accurately describe those dependencies, and that of the dependency manager (bundler) to handle those dependencies.

Not sure how related this is, but my reason is not only in app's code, it is also related to gems using YAML configuration since there are some new limitations enabled by default.

[marcandre]

If all you need to do to fix this problem currently is to lock rspec-rails to ~> 3.9 it is still super simple solution (considering how old Ruby is used here).

Right, and the proposal here does not solve anything for rspec-rails 4.0 or Ruby 2.2, of course, and that's fine :-) It only aims to make it possible and easy in the future to remove invalid gem manifests from the circulation, without too much impact.

[marcandre]

BTW, I was curious about the equivalent in elixir world. hex appears to have no yank equivalent, but can "retire" a package; that's informational only and will still resolve.

[sonalkr132]

I haven't gone through the issues raised in this thread, however, I want to add that we do plan to implement a soft yank-like feature for security advisories. Prev work rubygems/rubygems#2319

[pirj]

@sonalkr132 Nice, thanks for the heads-up.
I'm not a security specialist, but to me it's related to risk management, and whether to allow bundling certain gems or not should be a choice. E.g. for an app in an alpha stage with no public users, or for an internal-only apps or tools, even severe security vulnerabilities are not a blocker.
Feel free to skip the rest of the discussion, as there is only one security-related comment here.

My reasoning for soft-yanking was different - installing a certain gem version will most probably result in app errors. Or dependent gem CI builds would break.

[JonRowe]

Hex's retire is basically this feature, it'd be nice if there was a way to make a gem retirable to new installs, but not existing installs, e.g. bundler rejects adding a "retired" gem to a lock file, but if its present already will only warn (loudly if needed) about it, or would require a --install-despite-retirement etc flag for peoples build purposes

[pirj]

Any conclusion? I can send an RFC PR, but would like to know that it's preliminarily agreed on.
I see two options:

1. Introduce soft-yanking as a new feature.
2. Introduce soft-yanking for a wide audience, and restrict hard-yanking to the RubyGems team.

[jchestershopify]

As an alternative, can a hard yank instead include a suggested alternate version?

[pirj]

Another case where soft-yanking or yanking with an alternate version would help mitigate a loose dependency mistake.