Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

replace pyupgrade+isort+autoflake with ruff #105

Merged
merged 21 commits into from
Mar 11, 2024
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
5533619
replace isort&autoflake with ruff
jakkdl Mar 5, 2024
b50fba0
remove autoflake, remove last vestige of isort (autoflake depended on…
jakkdl Mar 6, 2024
94859af
""""fix"""" coverage
jakkdl Mar 6, 2024
4a02434
a bunch of codemods replaced with ruff
jakkdl Mar 6, 2024
61d63e3
add commented-out error codes for all checks that zac enabled in thei…
jakkdl Mar 6, 2024
91bbf5b
enable B011, simplify ShedFixers.leave_Assert
jakkdl Mar 6, 2024
cfd5c83
comments after researching PT018
jakkdl Mar 6, 2024
b2828f0
comments after researching unnecessary len()/bool()
jakkdl Mar 6, 2024
7e91f6f
comments after researching SIM117 vs removed_nested_with codemod
jakkdl Mar 6, 2024
fa359a5
replace collapse_isinstance_checks with SIM101
jakkdl Mar 6, 2024
33e934b
fix --check fail, revert leave_Assert changes since ruff also doesn't…
jakkdl Mar 6, 2024
bac8ede
pragma: no cover
jakkdl Mar 6, 2024
bae73ab
oops. Though 3.8 seems totes broke dependency-wise.
jakkdl Mar 6, 2024
5b671ba
generate deps on 3.8, since libcst==1.2 requires 3.9+
jakkdl Mar 6, 2024
2e51c6b
add first draft of CODEMODS.md
jakkdl Mar 8, 2024
5b9e863
minor fixes for CODEMODS.md
jakkdl Mar 8, 2024
f59d4da
changes after review
jakkdl Mar 11, 2024
6569599
remove pyupgrade, enable rules in ruff instead.
jakkdl Mar 11, 2024
0667eae
update README.md
jakkdl Mar 11, 2024
5e49913
cheat ruff by using 'py' instead of 'python' to mark code blocks
jakkdl Mar 11, 2024
87e80cf
reformat some code blocks, hide some from shed by marking them with '…
jakkdl Mar 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .git-blame-ignore-revs
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# remove carriage returns
bfb380e9538ca1502055cade65e660ffbdba10c8
379 changes: 379 additions & 0 deletions CODEMODS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,379 @@
This document attempts to document all codemods that shed does.


## `leave_Assert`
Remove redundant `assert x` where x is a literal that always evaluate to `True`.
Replace `assert y` where y is a literal that always evaluates to `False`.

### Examples
#### input
```python
# truthy statements removed, with corresponding comments
assert True
assert "message" # rip
assert (1, 2)

# false statements replaced with raise AssertionError
assert None
assert ""
assert 0, "hello"

assert False, "this is the only case handled by ruff"

```
#### output
```python
# false statements replaced with raise AssertionError
raise AssertionError
raise AssertionError
raise AssertionError("hello")

raise AssertionError("this is the only case handled by ruff")
```

Full test data in tests/recorded/asserts.txt

Ruff supports autofixing B011, but that only covers the precise case of exactly replacing `assert False` with `raise AssertionError`. It does
jakkdl marked this conversation as resolved.
Show resolved Hide resolved

## `remove_pointless_parens_around_call`
Removes pointless parentheses wrapping a call.

### Examples
Full test data in `tests/recorded/parens.txt` and `tests/recorded/parens_with_comment.txt`
#### input
```py
(list("abc"))
([].append(1))
(["foo"].append("bar"))
(["foo"].append("bar"))
foo((list("abc")))
```

#### output
```py
list("abc")
[].append(1)
["foo"].append("bar")
["foo"].append("bar")
foo(list("abc"))
```


## `replace_unnecessary_nested_calls`
Resolves flake8-comprehension C414. Ruffs implementation currently breaks sorting stabilityin one case.
jakkdl marked this conversation as resolved.
Show resolved Hide resolved

### Examples
Full test data in `tests/recorded/comprehensions/C414.txt`
#### input
```py
list(tuple(iterable))
set(reversed(iterable))
sorted(reversed(iterable)) # unsafe to fix, but ruff does

# cases handled by our codemod, but not by ruff
sorted(sorted(iterable, reverse=True))
sorted(sorted(iterable, reverse=True))
sorted(sorted(iterable, reverse=True), reverse=False)
sorted(sorted(iterable, reverse=False), reverse=True)

# unsafe to fix
jakkdl marked this conversation as resolved.
Show resolved Hide resolved
sorted(sorted(iterable), key=int)
sorted(sorted(iterable, key=bool))
sorted(sorted(iterable, key=bool), key=int)
```
#### output
```py
list(iterable)
set(iterable)
sorted(reversed(iterable)) # unsafe to fix, but ruff does

# cases handled by our codemod, but not by ruff
sorted(iterable)
sorted(iterable)
sorted(iterable, reverse=False)
sorted(iterable, reverse=True)

# unsafe to fix
sorted(sorted(iterable), key=int)
sorted(sorted(iterable, key=bool))
sorted(sorted(iterable, key=bool), key=int)
```


## `replace_unnecessary_subscript_reversal`
Fix flake8-comprehensions C415.

Unnecessary subscript reversal of iterable within `[reversed/set/sorted]()`.
ruff does not support autofixing of C415

#### input
```py
set(iterable[::-1])
sorted(iterable[::-1])
reversed(iterable[::-1])

set(iterable[None:])
set(iterable[:None:1])
set(iterable[None:None:])
```
#### output
```py
set(iterable)
sorted(iterable)
reversed(iterable)

set(iterable)
set(iterable)
set(iterable)
```


## `reorder_union_literal_contents_none_last`
Puts `None` last in a subscript of `Union` or `Literal`

Test data in tests/recorded/flatten_literal.txt and tests/recorded/reorder_none.txt

#### input
```py
var3: Optional[None, int, float]
def g(x: Union[None, float] = None):
pass
foo: set[None, int]

```
#### output
```py
var3: Optional[int, float, None]
def g(x: Union[int, None] = None):
pass
foo: set[int, None]
```

## `reorder_merge_optional_union`
Turn `Union[..., None]` into `Optional[Union[...]]`.

Test data in tests/recorded/flatten_union.txt
#### input
```py
Union[int, str, None]
Union[int, None, str]
```
#### output
```py
Optional[Union[int, str]]
Optional[Union[int, str]]
```

## `reorder_union_operator_contents_none_last`
Reorders binary-operator type unions to have `None` last.

Test data in tests/recorded/union_op_none_last.txt
#### input
```py
None | int # not a type annotation
var: None | int
var2: bool | None | float
var3: float | None | bool
```
#### output
```py
None | int # not a type annotation
var: int | None
var2: bool | float | None
var3: float | bool | None
```

## [Not Implemented] sort union types
Fully sort types in union/optional, so as to standardize their ordering and make it easier to see if two type annotations are in fact the same. This was never implemented, it was deemed to controversial as shed does not allow disabling specific checks, but may be of interest to ruff.

#### input
```py
k: Union[int, float]
l: Union[int, float]
m: float | bool
n: Optional[str, MyOtherType, bool]
```

#### output
```py
k: Union[float, int]
l: Union[float, int]
m: bool | float
n: Optional[MyOtherType, bool, str]
```

## `flatten_literal_subscript`
Flattens a `Literal` inside a `Literal`.

Test data in tests/recorded/flatten_literal.txt
#### input
```py
Literal[1, Literal[2, 3]]
```
#### output
```py
Literal[1, 2, 3]
```

## `flatten_union_subscript`
Flattens an `Optional`/`Union` inside a `Union`.

Test data in tests/recorded/flatten_literal.txt
#### input
```py
Union[int, Optional[str], bool]
Union[int, Optional[str], None]
Union[Union[int, float], str]
Union[int, Literal[1, 2]] # this should not change
Union[int, Literal[1, 2], Optional[str]]
Union[int, str, None]
```
#### output
```py
Union[int, str, bool, None]
Union[int, str, None]
Union[int, float, str]
Union[int, Literal[1, 2]] # this should not change
Union[int, Literal[1, 2], str, None]
Union[int, str, None]
```

## `discard_empty_else_blocks`
An `else: pass` block can always simply be discarded. This should also remove `else: ...` blocks but that's not currently supported.

Test data in tests/recorded/empty-else.txt
#### input
```py
if foo:
...
else:
pass

while foo:
...
else:
pass

for _ in range(10):
...
else:
pass

try:
1 / 0
else:
pass
jakkdl marked this conversation as resolved.
Show resolved Hide resolved

if foo:
...
else:
...
```

#### output
```py
if foo:
...

while foo:
...

for _ in range(10):
...

try:
1 / 0

if foo:
...
else: # should ideally be removed
...
```

## `remove_lambda_indirection`
Removes useless lambda wrapper.

Test data in tests/recorded/lambda-unwrapping.txt
#### input
```py
lambda: self.func()
lambda x: foo(x)
lambda x, y, z: (t + u).math_call(x, y, z)
```

#### output
```py
self.func
foo
(t + u).math_call
```

## `split_assert_and`
Split `assert a and b` into `assert a` and `assert b`. This is supported by ruff, but we have much more sophisticated handling of comments.

Test data in tests/recorded/split_assert.txt

## `remove_unnecessary_call_*`
Removes unnecessary len/bool calls in tests, autofixes PIE787.

No check for this in ruff; SIM103 needless-bool is specifically about returns.

Test data in tests/recorded/pie788_no_bool.txt and tests/recorded/pie788_no_len.txt

#### input
```py
if len(foo):
...
while bool(foo):
...
print(5 if len(foo) else 7)
```
#### output
```py
if foo:
...
while foo:
...
print(5 if foo else 7)
```

## `split_nested_with`
Ruff does this, but only if it wouldn't turn it into a multi-line with.

#### input
```py
with make_context_manager(1) as cm1:
with make_context_manager(2) as cm2:
pass
# Preserve this comment

with make_context_manager(1) as cm1, make_context_manager(2) as cm2:
with make_context_manager(3) as cm3:
pass
with make_context_manager(1) as cm1:
with make_context_manager(2) as cm2:
with make_context_manager(3) as cm3:
with make_context_manager(4) as cm4:
pass
```

#### output
```py
with make_context_manager(1) as cm1, make_context_manager(2) as cm2:
pass
# Preserve this comment
with (
make_context_manager(1) as cm1,
make_context_manager(2) as cm2,
make_context_manager(3) as cm3,
):
pass

with (
make_context_manager(1) as cm1,
make_context_manager(2) as cm2,
make_context_manager(3) as cm3,
make_context_manager(4) as cm4,
):
pass
```
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ or explicitly passed a list of files to format on the command-line.
## Features
`shed`...

- Runs [`autoflake`](https://pypi.org/project/autoflake/),
- Runs [`ruff`](https://pypi.org/project/ruff/),
to remove unused imports and variables
- Runs [`pyupgrade`](https://pypi.org/project/pyupgrade/),
with autodetected minimum version >= py38
Expand Down
Loading
Loading