MinerX.3 v1.13.0 Deal Making Testing #7465
Locked
jennijuju
started this conversation in
Show and tell
Replies: 13 comments 7 replies
-
|
Still seeing all proposed Estuary deals stuck at 0B: |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Beta Was this translation helpful? Give feedback.
-
Additional FYI: I have also set these new flags in my markets |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
see https://filecoinproject.slack.com/archives/C02G9S9T91Q/p1633619727299700 |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
The default settings are not compatible with the workload of estuary. SimultaneousTransfersForStorage should go to 200 or something like that. |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
I think each sp should decide on what this value based on their setup - how much workload can SP take is what matters not the client demand imho |
Beta Was this translation helpful? Give feedback.
-
|
Estuary opens more then 20 connections to a miner by default as far as I can see. There is no current way of limiting that. I you keep the 20 in place transfers crash and keep stuck at 0B |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
yes, i see the same - default does not apply. |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
After testing the I had 4 deals that had an When I canceled the 4 deals with So, the |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
@rjan90 , yes I see the same happening here. The transfers do seem to work if nothing is in the way :-) . Thanks, |
Beta Was this translation helpful? Give feedback.
-
Follow up Testing - 2021-10-24First of all, huge thanks to the fellows that have been helping testing out the new releases! Unfortunately, there are some client side graph-sync bug that is causing Estaury to not able to proceed a huge amount of data transfers properly. The team is currently investigating and working on the issue actively. While that is being worked on, we'd like to first confirm that the newly introduced feature - data transfer rate limiting(especially for storage deals) is working properly. This is to prevent your market node getting flood by Estuary deals' data transfer as well once things are working. Therefore, please do the following steps:
Please reply to this thread if:
|
Beta Was this translation helpful? Give feedback.
0 replies
-
|
lotus-miner version |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Re enabled Estuary - sometimes it works fine and then overnight found dozens of stuck deals with a "Message: graphsync request cancelled" message. |
Beta Was this translation helpful? Give feedback.
-
|
lotus-miner version MaxStagingDealsBytes = 1099511627776 |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
lotus-miner version: With: |
Beta Was this translation helpful? Give feedback.
2 replies
-
|
It seems to be working quite well: I got 8 deals, where 5 of them where transferring while the rest was waiting in queue (0B) When |
Beta Was this translation helpful? Give feedback.
-
|
This is great news! |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
0 replies
-
|
lotus-miner version market node config.toml After updating lotus version, I canceled all the stuck 7.4TiB transfers (log just before the update), so it was back to square one. Transfers started without problems from the 1st to the 9th and currently running, but on or after the 10th are stuck at 0B. I want to keep a close watch on how the state will change when the 9 transfers will be completed. 1st time to run the command lotus-miner info lotus-miner data-transfers list I saw Message: graphsync request canceled, but the transfer is still going on. 2nd time to run the command lotus-miner info lotus-miner data-transfers list |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
3rd time to run the command lotus-miner info lotus-miner data-transfers list |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Thanks to @hannahhoward, @dirkmc, @whyrusleeping, @arajasek and many others, we've got some exciting bug fixes that potentially resolve most of the top outstanding issues with online deals, especially with Estuary! So, update your market system with lotus v1.13.0-rc2 and start making deals!
🌟 New Feature and Improvement 🌟
Data Transfer Throttling
You can now set an ongoing data transfer limit for both storage and retrieval deals, separately!
In your market repo's
config.tomlfile underDealmakingsession, you can define the maximum number of parallel online data transfers for storage and retrievals by settingSimultaneousTransfersForStorageandSimultaneousTransfersForRetrievalaccordingly. You can also set these two value by setting the envvarLOTUS_DEALMAKING_SIMULTANEOUSTRANSFERSFORSTORAGEandLOTUS_DEALMAKING_SIMULTANEOUSTRANSFERSFORRETRIEVAL.The default value for both is set to 20.
Start Epoch Buffer
According to #5137, sometimes (soon-to-be)expired deals were queued in the sealing pipeline for new sectors and caused the new sectors to fail to seal as well. To resolve this issue, we have added a new
StartEpochSealingBufferconfiguration variable underDealmakingfor storage providers to set a buffer time of getting an accepted deal into a sector according to the deal's start epoch and their expected sealing time. This check is done prior to AP and sending out a PSD message. This value can also be set by envvarLOTUS_DEALMAKING_STARTEPOCHSEALINGBUFFER.The default value is set to 480 epochs.
We are hoping 🤫
Steps
12D3KooWGBWx9gyUFTVQcKMTenQMSyE2ad9m7c9fpjS4NMjoDien12D3KooWFrnuj5o3tx4fGD2ZVJRyDqTdzGnU3XYXmBbWbc8Hs8Nd12D3KooWNNuTcH39tagS4MyddgQP7jBNSHJn4cVuTKuY4bjNZ6cwlotus-miner --call-on-markets log set-level --system dt-impl --system dt-chanmon --system dt_graphsync --system markets --system data_transfer_network --system retrievaladapter --system retrieval --system stores --system markets-rtvl-reval --system graphsync_allocator debuglotus-miner info allif you run into any issue! If the log files are large, you can upload them inMinerX.3 - v1.13.0 Testing Logsfolder and link it in your issue report/this discussion/slack message/..Beta Was this translation helpful? Give feedback.
All reactions