feat: move unit tests from Circleci to Github actions #25570

itsyoboieltr · 2024-06-27T18:29:47Z

Description

This PR moves unit tests from Circleci to Github actions. This is needed to pass the Jest export to SonarCloud in order finalize the SonarCloud integration. The reason why we want to use Github action rather than Circleci is because Sonarcloud integration happens in a Github action and we cannot pass data (i.e. the artifact) from Circleci to Github action.

Related issues

Fixes: https://github.com/MetaMask/MetaMask-planning/issues/2449

Manual testing steps

Seeing unit tests pass in GitHub actions instead of Circleci

Screenshots/Recordings

Not applicable

Pre-merge author checklist

I've followed MetaMask Contributor Docs and MetaMask Extension Coding Standards.
I've completed the PR template to the best of my ability
I’ve included tests if applicable
I’ve documented my code using JSDoc format if applicable
I’ve applied the right labels on the PR (see labeling guidelines). Not required for external contributors.

Pre-merge reviewer checklist

I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

github-actions · 2024-06-27T18:30:02Z

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

codecov · 2024-06-28T18:56:39Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 69.77%. Comparing base (56c9dc2) to head (d1be009).
Report is 14 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop   #25570      +/-   ##
===========================================
+ Coverage    69.67%   69.77%   +0.09%     
===========================================
  Files         1366     1376      +10     
  Lines        48231    48388     +157     
  Branches     13304    13346      +42     
===========================================
+ Hits         33604    33758     +154     
- Misses       14627    14630       +3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

metamaskbot · 2024-06-28T19:08:11Z

Builds ready [a399875]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
- content-script: 0
- offscreen: 0
- ui: 0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (46 ± 3 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	66	97	80	8	4
		domContentLoaded	8	17	10	2	1
		load	38	64	46	7	3
		domInteractive	8	17	10	2	1

Bundle size diffs

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

metamaskbot · 2024-06-28T21:37:02Z

Builds ready [ce18831]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
- content-script: 0
- offscreen: 0
- ui: 0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (225 ± 246 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	71	114	89	12	6
		domContentLoaded	9	36	12	6	3
		load	43	1778	225	512	246
		domInteractive	9	36	12	6	3

Bundle size diffs

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

I would be honestly surprised if this worked

metamaskbot · 2024-07-02T13:19:09Z

Builds ready [9711df5]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
- content-script: 0
- offscreen: 0
- ui: 0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (70 ± 11 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	69	145	100	21	10
		domContentLoaded	9	56	25	13	6
		load	43	118	70	22	11
		domInteractive	9	56	25	13	6

Bundle size diffs

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

davidmurdoch

just a partial review with some questions before I look more closely.

.circleci/config.yml

.github/workflows/run-unit-tests.yml

metamaskbot · 2024-07-03T15:33:19Z

Builds ready [6f6fbf6]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
- content-script: 0
- offscreen: 0
- ui: 0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (133 ± 152 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	62	206	96	31	15
		domContentLoaded	9	55	23	12	6
		load	39	1508	133	316	152
		domInteractive	9	55	23	12	6

Bundle size diffs

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

.github/workflows/run-unit-tests.yml

metamaskbot · 2024-07-03T16:51:18Z

Builds ready [15797c2]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
- content-script: 0
- offscreen: 0
- ui: 0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (356 ± 334 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	68	305	114	64	31
		domContentLoaded	9	98	28	21	10
		load	39	2237	356	696	334
		domInteractive	9	98	28	21	10

Bundle size diffs

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

davidmurdoch

As discussed in slack, let's follow up this PR with an investigation into making it go faster (to match the speeds at which Circleci ran the same jobs).

But in case it takes a while to do that work: can you add a new GitHub Issue and a comment in this file linking to the issue, explaining that we should look into making it go faster?

I just don't want people copying this work in the (near) future, not noticing the slow down, and suddenly our tests take hours to pass CI. Hopefully a comment in here will help prevent that.

relevant:

HowardBraham · 2024-07-03T18:30:44Z

As discussed in slack, let's follow up this PR with an investigation into making it go faster (to match the speeds at which Circleci ran the same jobs).

There are 2 reasons it's slower than CircleCI:

It's not using the yarn cache from CircleCI, so it has to redo the full work of CircleCI's prep-deps
CircleCI is running with parallelism 8 (which is honestly too high)

itsyoboieltr · 2024-07-04T12:11:31Z

There are 2 reasons it's slower than CircleCI:

It's not using the yarn cache from CircleCI, so it has to redo the full work of CircleCI's prep-deps

CircleCI is running with parallelism 8 (which is honestly too high)

@HowardBraham Do you think the first point is really a reason for the slow down? I do not necessarily think so, since it is using the yarn cache from GitHub, meaning it does not need to redo the full work.

The second point about parallelism is true, we do need a fix for that.

sonarcloud · 2024-07-04T13:33:53Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

davidmurdoch

⭐⭐⭐⭐⭐
Five stars!
Would review again.

metamaskbot · 2024-07-04T14:46:24Z

Builds ready [d1be009]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
- content-script: 0
- offscreen: 0
- ui: 0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (177 ± 216 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	76	135	107	18	9
		domContentLoaded	9	61	31	17	8
		load	41	2134	177	449	216
		domInteractive	9	61	30	17	8

Bundle size diffs

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

HowardBraham · 2024-07-04T21:25:17Z

@itsyoboieltr In the prep-deps step, CircleCI does a persist_to_workspace on the node_modules folder. So in future steps, it does not run yarn install at all, it just does attach_workspace, which brings back the node_modules. This step tends to take 14 seconds.

Gudahtt · 2024-07-10T11:44:21Z

.github/workflows/run-unit-tests.yml

+name: Run unit tests
+
+on:
+  pull_request:


We need to add a trigger for develop and master here as well. These long-running branches will not have PRs associated with them, but it's very important that we know that tests are passing on them.

feat: run-unit-tests

d39b4a5

itsyoboieltr self-assigned this Jun 27, 2024

metamaskbot added the team-extension-platform label Jun 27, 2024

itsyoboieltr added 2 commits June 28, 2024 20:35

fix: add corepack enable

f353ab5

fix: corepack enable

a399875

itsyoboieltr added 3 commits June 28, 2024 23:01

fix: remove unit tests from config

b678880

fix: add pr action types

9b8917e

fix: setup nodejs

ce18831

itsyoboieltr added 4 commits July 1, 2024 13:13

Merge branch 'develop' into move-unit-tests-to-github-actions

766a339

feat: add codecov upload

335744f

fix: test

697f8ec

I would be honestly surprised if this worked

fix: add token and fail on error

6b03bd3

itsyoboieltr marked this pull request as ready for review July 1, 2024 16:00

itsyoboieltr requested review from kumavis and a team as code owners July 1, 2024 16:00

itsyoboieltr added 3 commits July 2, 2024 14:35

Merge branch 'develop' into move-unit-tests-to-github-actions

c082f24

Merge branch 'develop' into move-unit-tests-to-github-actions

0cc1a57

Update config.yml

9711df5

itsyoboieltr requested a review from davidmurdoch July 2, 2024 16:30

davidmurdoch requested changes Jul 2, 2024

View reviewed changes

.circleci/config.yml Outdated Show resolved Hide resolved

.github/workflows/run-unit-tests.yml Outdated Show resolved Hide resolved

.github/workflows/run-unit-tests.yml Outdated Show resolved Hide resolved

.github/workflows/run-unit-tests.yml Outdated Show resolved Hide resolved

itsyoboieltr added 4 commits July 3, 2024 16:18

feat: setup-environment reusable workflow

042fe6f

fix: checkout

4b3f0cd

Update run-unit-tests.yml

ce890f9

fix: composite action

97eb64e

itsyoboieltr added 3 commits July 3, 2024 16:51

Update action.yml

6814ed1

Update action.yml

c6a4933

Update config.yml

6f6fbf6

itsyoboieltr added 2 commits July 3, 2024 17:35

Update action.yml

32cdf03

Update action.yml

62c2e93

davidmurdoch reviewed Jul 3, 2024

View reviewed changes

.github/workflows/run-unit-tests.yml Outdated Show resolved Hide resolved

itsyoboieltr added 3 commits July 3, 2024 18:03

Update run-unit-tests.yml

e63bc82

Merge branch 'develop' into move-unit-tests-to-github-actions

0438fc4

Update run-unit-tests.yml

15797c2

davidmurdoch requested changes Jul 3, 2024

View reviewed changes

itsyoboieltr mentioned this pull request Jul 4, 2024

Make Jest unit tests run faster in GitHub actions #25680

Closed

9 tasks

fix: add warning

d1be009

davidmurdoch approved these changes Jul 4, 2024

View reviewed changes

danjm approved these changes Jul 4, 2024

View reviewed changes

itsyoboieltr merged commit 2050886 into develop Jul 5, 2024
71 checks passed

itsyoboieltr deleted the move-unit-tests-to-github-actions branch July 5, 2024 14:26

github-actions bot locked and limited conversation to collaborators Jul 5, 2024

metamaskbot added the release-12.2.0 Issue or pull request that will be included in release 12.2.0 label Jul 5, 2024

Gudahtt reviewed Jul 10, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: move unit tests from Circleci to Github actions #25570

feat: move unit tests from Circleci to Github actions #25570

itsyoboieltr commented Jun 27, 2024 •

edited by github-actions bot

Loading

github-actions bot commented Jun 27, 2024

codecov bot commented Jun 28, 2024 •

edited

Loading

metamaskbot commented Jun 28, 2024

metamaskbot commented Jun 28, 2024

metamaskbot commented Jul 2, 2024

davidmurdoch left a comment

metamaskbot commented Jul 3, 2024

metamaskbot commented Jul 3, 2024

davidmurdoch left a comment

HowardBraham commented Jul 3, 2024

itsyoboieltr commented Jul 4, 2024 •

edited

Loading

sonarcloud bot commented Jul 4, 2024

davidmurdoch left a comment

metamaskbot commented Jul 4, 2024

HowardBraham commented Jul 4, 2024

Gudahtt Jul 10, 2024

feat: move unit tests from Circleci to Github actions #25570

feat: move unit tests from Circleci to Github actions #25570

Conversation

itsyoboieltr commented Jun 27, 2024 • edited by github-actions bot Loading

Description

Related issues

Manual testing steps

Screenshots/Recordings

Pre-merge author checklist

Pre-merge reviewer checklist

github-actions bot commented Jun 27, 2024

codecov bot commented Jun 28, 2024 • edited Loading

Codecov Report

metamaskbot commented Jun 28, 2024

metamaskbot commented Jun 28, 2024

metamaskbot commented Jul 2, 2024

davidmurdoch left a comment

Choose a reason for hiding this comment

metamaskbot commented Jul 3, 2024

metamaskbot commented Jul 3, 2024

davidmurdoch left a comment

Choose a reason for hiding this comment

HowardBraham commented Jul 3, 2024

itsyoboieltr commented Jul 4, 2024 • edited Loading

sonarcloud bot commented Jul 4, 2024

Quality Gate passed

davidmurdoch left a comment

Choose a reason for hiding this comment

metamaskbot commented Jul 4, 2024

HowardBraham commented Jul 4, 2024

Gudahtt Jul 10, 2024

Choose a reason for hiding this comment

itsyoboieltr commented Jun 27, 2024 •

edited by github-actions bot

Loading

codecov bot commented Jun 28, 2024 •

edited

Loading

itsyoboieltr commented Jul 4, 2024 •

edited

Loading