Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

renovate: add test that should fail #137

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

renovate: add test that should fail #137

wants to merge 2 commits into from

Conversation

roobre
Copy link
Contributor

@roobre roobre commented Feb 2, 2023

Meant as an example rather than the actual refactor.

Renovate tests are currently missing some cases that should making them fail.

Moreover, when they fail, the output is very unreadable.

For example, consider this change:

diff --git a/src/changelog/sources/renovate/source_test.go b/src/changelog/sources/renovate/source_test.go
index 0471894..f4bf520 100644
--- a/src/changelog/sources/renovate/source_test.go
+++ b/src/changelog/sources/renovate/source_test.go
@@ -85,7 +85,7 @@ func TestSource_Source(t *testing.T) {
 * Bum chart's version
 * fix typo in the common library`},
 				{Message: "chore(deps): update helm release common-library-2 to v0.0.4"},
-				{Message: "chore(deps): update helm release common-library-3/library to v1.2.3"},
+				{Message: "bogus commit"},
 				{Message: "chore(deps): update helm release common-library-4 to v10.20.30"},
 				{Message: "chore(deps): update helm release common-library-5 to v1.1.2-prerelease+meta"},
 				{Message: "chore(deps): update helm release common-library-6 to v1.1.2+meta"},

The output of the test would be gigantic, because all expectations are offset by one:

Gigantic code block 
?   	github.com/newrelic/release-toolkit	[no test files]
?   	github.com/newrelic/release-toolkit/src/app	[no test files]
ok  	github.com/newrelic/release-toolkit/src/app/common	(cached)
ok  	github.com/newrelic/release-toolkit/src/app/generate	(cached)
?   	github.com/newrelic/release-toolkit/src/app/gha	[no test files]
ok  	github.com/newrelic/release-toolkit/src/app/isheld	(cached)
ok  	github.com/newrelic/release-toolkit/src/app/link	(cached)
ok  	github.com/newrelic/release-toolkit/src/app/nextversion	(cached)
ok  	github.com/newrelic/release-toolkit/src/app/render	(cached)
ok  	github.com/newrelic/release-toolkit/src/app/update	(cached)
ok  	github.com/newrelic/release-toolkit/src/app/validate	(cached)
ok  	github.com/newrelic/release-toolkit/src/bump	(cached)
ok  	github.com/newrelic/release-toolkit/src/bumper	(cached)
ok  	github.com/newrelic/release-toolkit/src/changelog	(cached)
ok  	github.com/newrelic/release-toolkit/src/changelog/linker	(cached)
ok  	github.com/newrelic/release-toolkit/src/changelog/linker/mapper	(cached)
ok  	github.com/newrelic/release-toolkit/src/changelog/renderer	(cached)
ok  	github.com/newrelic/release-toolkit/src/changelog/sources/dependabot	(cached)
ok  	github.com/newrelic/release-toolkit/src/changelog/sources/markdown	(cached)
?   	github.com/newrelic/release-toolkit/src/changelog/sources/markdown/headingdoc	[no test files]
ok  	github.com/newrelic/release-toolkit/src/changelog/sources/markdown/merger	(cached)
time="2023-02-02T12:08:09+01:00" level=warning msg="Renovate could not extract updated version from \"chore(deps): update helm release common-library-3\""
time="2023-02-02T12:08:09+01:00" level=warning msg="Renovate could not extract updated version from \"chore(deps): update integrations (#124)\""
--- FAIL: TestSource_Source (0.00s)
    --- FAIL: TestSource_Source/Matching_commits (0.00s)
        source_test.go:256: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:256
            	Error:      	Not equal: 
            	            	expected: 41
            	            	actual  : 40
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-3/library"
            	            	actual  : "common-library-4"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-3/library
            	            	+common-library-4
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.2.3"
            	            	actual  : "10.20.30"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.2.3
            	            	+10.20.30
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-4"
            	            	actual  : "common-library-5"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-4
            	            	+common-library-5
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "10.20.30"
            	            	actual  : "1.1.2-prerelease+meta"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-10.20.30
            	            	+1.1.2-prerelease+meta
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-5"
            	            	actual  : "common-library-6"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-5
            	            	+common-library-6
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.1.2-prerelease+meta"
            	            	actual  : "1.1.2+meta"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.1.2-prerelease+meta
            	            	+1.1.2+meta
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-6"
            	            	actual  : "common-library-7"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-6
            	            	+common-library-7
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.1.2+meta"
            	            	actual  : "1.1.2+meta-valid"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.1.2+meta
            	            	+1.1.2+meta-valid
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-7"
            	            	actual  : "common-library-8"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-7
            	            	+common-library-8
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.1.2+meta-valid"
            	            	actual  : "1.0.0-alpha"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.1.2+meta-valid
            	            	+1.0.0-alpha
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-8"
            	            	actual  : "common-library-9"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-8
            	            	+common-library-9
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.0.0-alpha"
            	            	actual  : "1.0.0-alpha.beta.1"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.0.0-alpha
            	            	+1.0.0-alpha.beta.1
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-9"
            	            	actual  : "common-library-10"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-9
            	            	+common-library-10
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.0.0-alpha.beta.1"
            	            	actual  : "1.0.0-alpha.1"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.0.0-alpha.beta.1
            	            	+1.0.0-alpha.1
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-10"
            	            	actual  : "common-library-11"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-10
            	            	+common-library-11
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.0.0-alpha.1"
            	            	actual  : "1.0.0-alpha0.valid"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.0.0-alpha.1
            	            	+1.0.0-alpha0.valid
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-11"
            	            	actual  : "common-library-12"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-11
            	            	+common-library-12
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.0.0-alpha0.valid"
            	            	actual  : "1.0.0-alpha.0"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.0.0-alpha0.valid
            	            	+1.0.0-alpha.0
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-12"
            	            	actual  : "common-library-13"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-12
            	            	+common-library-13
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.0.0-alpha.0"
            	            	actual  : "1.0.0-alpha-a.b-c-somethinglong+build.1-aef.1-its-okay"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.0.0-alpha.0
            	            	+1.0.0-alpha-a.b-c-somethinglong+build.1-aef.1-its-okay
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-13"
            	            	actual  : "common-library-14"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-13
            	            	+common-library-14
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.0.0-alpha-a.b-c-somethinglong+build.1-aef.1-its-okay"
            	            	actual  : "2.0.0-rc.1+build.123"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.0.0-alpha-a.b-c-somethinglong+build.1-aef.1-its-okay
            	            	+2.0.0-rc.1+build.123
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-14"
            	            	actual  : "common-library-15"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-14
            	            	+common-library-15
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "2.0.0-rc.1+build.123"
            	            	actual  : "1.2.3-SNAPSHOT-123"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-2.0.0-rc.1+build.123
            	            	+1.2.3-SNAPSHOT-123
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-15"
            	            	actual  : "common-library-16"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-15
            	            	+common-library-16
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.2.3-SNAPSHOT-123"
            	            	actual  : "2.0.0+build.1848"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.2.3-SNAPSHOT-123
            	            	+2.0.0+build.1848
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-16"
            	            	actual  : "common-library-17"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-16
            	            	+common-library-17
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "2.0.0+build.1848"
            	            	actual  : "2.0.1-alpha.1227"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-2.0.0+build.1848
            	            	+2.0.1-alpha.1227
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-17"
            	            	actual  : "common-library-18"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-17
            	            	+common-library-18
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "2.0.1-alpha.1227"
            	            	actual  : "1.0.0-alpha+beta"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-2.0.1-alpha.1227
            	            	+1.0.0-alpha+beta
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-18"
            	            	actual  : "common-library-19"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-18
            	            	+common-library-19
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.0.0-alpha+beta"
            	            	actual  : "1.2.3----RC-SNAPSHOT.12.9.1"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.0.0-alpha+beta
            	            	+1.2.3----RC-SNAPSHOT.12.9.1
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-19"
            	            	actual  : "common-library-20"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-19
            	            	+common-library-20
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.2.3----RC-SNAPSHOT.12.9.1"
            	            	actual  : "1.0.0+0.build.1-rc.10000aaa-kk-0.1"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.2.3----RC-SNAPSHOT.12.9.1
            	            	+1.0.0+0.build.1-rc.10000aaa-kk-0.1
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-20"
            	            	actual  : "common-library-21"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-20
            	            	+common-library-21
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.0.0+0.build.1-rc.10000aaa-kk-0.1"
            	            	actual  : "999999.999999.999999"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.0.0+0.build.1-rc.10000aaa-kk-0.1
            	            	+999999.999999.999999
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-21"
            	            	actual  : "rust crate i-do-not-exist"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-21
            	            	+rust crate i-do-not-exist
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "999999.999999.999999"
            	            	actual  : "1.2.3"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-999999.999999.999999
            	            	+1.2.3
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "rust crate i-do-not-exist"
            	            	actual  : "noprefix"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-rust crate i-do-not-exist
            	            	+noprefix
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "noprefix"
            	            	actual  : "name with spaces"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-noprefix
            	            	+name with spaces
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "name with spaces"
            	            	actual  : "fancy-module"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-name with spaces
            	            	+fancy-module
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "fancy-module"
            	            	actual  : "module"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-fancy-module
            	            	+module
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "module"
            	            	actual  : "newrelic/infrastructure-bundle"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-module
            	            	+newrelic/infrastructure-bundle
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.2.3"
            	            	actual  : "2.7.6"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.2.3
            	            	+2.7.6
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:264: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:264
            	Error:      	Not equal: 
            	            	expected: changelog.EntryMeta{Author:"", PR:"", Commit:""}
            	            	actual  : changelog.EntryMeta{Author:"", PR:"280", Commit:""}
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -2,3 +2,3 @@
            	            	  Author: (string) "",
            	            	- PR: (string) "",
            	            	+ PR: (string) (len=3) "280",
            	            	  Commit: (string) ""
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "newrelic/infrastructure-bundle"
            	            	actual  : "github.com/google/go-github/v39"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-newrelic/infrastructure-bundle
            	            	+github.com/google/go-github/v39
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "2.7.6"
            	            	actual  : "39.2.0"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-2.7.6
            	            	+39.2.0
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:264: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:264
            	Error:      	Not equal: 
            	            	expected: changelog.EntryMeta{Author:"", PR:"280", Commit:""}
            	            	actual  : changelog.EntryMeta{Author:"", PR:"123", Commit:""}
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -2,3 +2,3 @@
            	            	  Author: (string) "",
            	            	- PR: (string) (len=3) "280",
            	            	+ PR: (string) (len=3) "123",
            	            	  Commit: (string) ""
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "github.com/google/go-github/v39"
            	            	actual  : "newrelic/infrastructure"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-github.com/google/go-github/v39
            	            	+newrelic/infrastructure
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "39.2.0"
            	            	actual  : "1.20.5"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-39.2.0
            	            	+1.20.5
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:264: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:264
            	Error:      	Not equal: 
            	            	expected: changelog.EntryMeta{Author:"", PR:"123", Commit:""}
            	            	actual  : changelog.EntryMeta{Author:"", PR:"125", Commit:""}
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -2,3 +2,3 @@
            	            	  Author: (string) "",
            	            	- PR: (string) (len=3) "123",
            	            	+ PR: (string) (len=3) "125",
            	            	  Commit: (string) ""
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "newrelic/infrastructure"
            	            	actual  : "integrations"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-newrelic/infrastructure
            	            	+integrations
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:262: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:262
            	Error:      	Expected nil, but got: &semver.Version{major:1, minor:20, patch:5, pre:"", metadata:"", original:"v1.20.5"}
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:264: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:264
            	Error:      	Not equal: 
            	            	expected: changelog.EntryMeta{Author:"", PR:"125", Commit:""}
            	            	actual  : changelog.EntryMeta{Author:"", PR:"124", Commit:""}
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -2,3 +2,3 @@
            	            	  Author: (string) "",
            	            	- PR: (string) (len=3) "125",
            	            	+ PR: (string) (len=3) "124",
            	            	  Commit: (string) ""
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "integrations"
            	            	actual  : "aquasecurity/trivy-action"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-integrations
            	            	+aquasecurity/trivy-action
            	Test:       	TestSource_Source/Matching_commits
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x51fd39]

goroutine 9 [running]:
testing.tRunner.func1.2({0x61e940, 0x821f90})
	/usr/lib/go/src/testing/testing.go:1396 +0x24e
testing.tRunner.func1()
	/usr/lib/go/src/testing/testing.go:1399 +0x39f
panic({0x61e940, 0x821f90})
	/usr/lib/go/src/runtime/panic.go:884 +0x212
github.com/Masterminds/semver.(*Version).String(0x0)
	/home/roobre/.go/pkg/mod/github.com/!masterminds/[email protected]/version.go:113 +0x59
github.com/newrelic/release-toolkit/src/changelog/sources/renovate_test.TestSource_Source.func1(0xc0001d01a0)
	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260 +0x385
testing.tRunner(0xc0001d01a0, 0xc00002f100)
	/usr/lib/go/src/testing/testing.go:1446 +0x10b
created by testing.(*T).Run
	/usr/lib/go/src/testing/testing.go:1493 +0x35f
FAIL	github.com/newrelic/release-toolkit/src/changelog/sources/renovate	0.006s
ok  	github.com/newrelic/release-toolkit/src/git	(cached)
?   	github.com/newrelic/release-toolkit/src/hack	[no test files]
ok  	github.com/newrelic/release-toolkit/src/version	(cached)
FAIL

This makes very hard to add new tests to the suite, as if the test does not work for one commit, all the others will fail as well. I think we should consider refactoring this test suite so it looks more like Dependabot's, which does not have this problem.

@roobre
Copy link
Contributor Author

roobre commented Feb 2, 2023

@alvarocabanas If you could take a look at this it would be awesome ❤️

@alvarocabanas alvarocabanas marked this pull request as ready for review February 22, 2023 08:02
alvarocabanas
alvarocabanas approved these changes Feb 22, 2023
View reviewed changes
Copy link
Contributor

@alvarocabanas alvarocabanas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

roobre
roobre commented Mar 13, 2023
View reviewed changes
Copy link
Contributor Author

@roobre roobre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks awesome @alvarocabanas, thanks a lot!

Left a quick doubt and after this we can merge this :)

roobre
roobre commented Mar 13, 2023
View reviewed changes
src/changelog/sources/renovate/source_test.go
@@ -41,9 +41,8 @@ func (c *commitsGetterMock) Commits(_ string) ([]git.Commit, error) {
return commits, nil
}

//nolint:funlen
//nolint:funlen,paralleltest,maintidx
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we getting rid of paralleltest? If this is unavoidable I'd love to see a short explanation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alvarocabanas alvarocabanas alvarocabanas approved these changes

Assignees

@alvarocabanas alvarocabanas

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@roobre @alvarocabanas