Batch plan

Message boards : News : Batch plan
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile Eric Driver
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 8 Jul 11
Posts: 1045
Credit: 125,908,995
RAC: 110,673
Message 2568 - Posted: 9 Oct 2019, 0:33:36 UTC

I know some of you like to know the batch plan, so here it is...

Batch 13x270 is nearing completion for subfield 3. After that, we will spend a week or two picking some of the low hanging fruit on subfield 7. Then it's back to subfield 3, batch 13x271. This is a very big batch, so I expect it to last until the end of December, possibly into January.
ID: 2568 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Mark1692

Send message
Joined: 8 Oct 19
Posts: 1
Credit: 24,632
RAC: 0
Message 2569 - Posted: 9 Oct 2019, 21:26:41 UTC - in response to Message 2568.  

seti@home graphics are not working....what am i doing wrong?
ID: 2569 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 28 Oct 11
Posts: 148
Credit: 122,940,599
RAC: 70,626
Message 2570 - Posted: 9 Oct 2019, 22:20:59 UTC - in response to Message 2569.  

seti@home graphics are not working....what am i doing wrong?
Posting in the wrong forum.

Try https://setiathome.berkeley.edu/forum_help_desk.php - but next time, please give details of your operating system, BOINC version, and what you did prior to the change in behaviour.
ID: 2570 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Vitaly

Send message
Joined: 5 Jan 13
Posts: 31
Credit: 7,320,272
RAC: 17,393
Message 2571 - Posted: 10 Oct 2019, 8:51:19 UTC - in response to Message 2570.  

Are DS13x271 tasks short (similar to DS13x270)?
ID: 2571 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Eric Driver
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 8 Jul 11
Posts: 1045
Credit: 125,908,995
RAC: 110,673
Message 2572 - Posted: 10 Oct 2019, 15:48:35 UTC - in response to Message 2571.  

Are DS13x271 tasks short (similar to DS13x270)?


No, they will be about twice as long. DS13x270 took about 10 days, so DS13x271 should take about 85 days, assuming FLOPS stay the same.
ID: 2572 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 28 Oct 11
Posts: 148
Credit: 122,940,599
RAC: 70,626
Message 2573 - Posted: 10 Oct 2019, 17:38:50 UTC - in response to Message 2572.  

Which raises the problem of the transition back to long tasks after our machines have adapted to the short runs from subfield 7. You may get a blip of runtime exceeded errors again, followed by another blip when the first resends try to run in half the time. Might be wise to cut down the 'maximum tasks in progress' for a while when the first long tasks are ready to flow out.
ID: 2573 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Eric Driver
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 8 Jul 11
Posts: 1045
Credit: 125,908,995
RAC: 110,673
Message 2574 - Posted: 10 Oct 2019, 19:43:00 UTC - in response to Message 2573.  

Which raises the problem of the transition back to long tasks after our machines have adapted to the short runs from subfield 7. You may get a blip of runtime exceeded errors again, followed by another blip when the first resends try to run in half the time. Might be wise to cut down the 'maximum tasks in progress' for a while when the first long tasks are ready to flow out.


Ok. I reduced the "max_wus_in_progress" parameter just to be safe. But just to clarify, the sf7 tasks will average about .7 hours and the future 13x271 tasks average about 1 hour, so they are actually pretty close in time. For comparison the 13x270 which recently completed (just waiting on a few stragglers to trickle in) averaged .5 hours. All these times are from the same 3.6 GHz computer, running single threaded. If you run hyper-threaded, you can double the times for a better runtime estimate.
ID: 2574 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Eric Driver
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 8 Jul 11
Posts: 1045
Credit: 125,908,995
RAC: 110,673
Message 2613 - Posted: 1 Jan 2020, 22:41:38 UTC - in response to Message 2568.  

As predicted, sf3 DS13x271 would take until the end of December. There are about 30k WUs still outstanding, which should take about 2 weeks to trickle in. In the meantime, the search has progressed to DS14x270. The current plan is to stay on subfield 3 for the next several months and knock out DS14 and DS15.
ID: 2613 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Speedy51

Send message
Joined: 13 Apr 19
Posts: 13
Credit: 2,152,705
RAC: 9,561
Message 2616 - Posted: 3 Jan 2020, 20:46:04 UTC - in response to Message 2613.  

Appreciate the update. I am assuming going off past experience DS15×271 will take about the same amount of time as DS13×271 if not longer simply because it has the same number of work units in the batch?
ID: 2616 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Eric Driver
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 8 Jul 11
Posts: 1045
Credit: 125,908,995
RAC: 110,673
Message 2617 - Posted: 4 Jan 2020, 3:24:01 UTC - in response to Message 2616.  

Appreciate the update. I am assuming going off past experience DS15×271 will take about the same amount of time as DS13×271 if not longer simply because it has the same number of work units in the batch?


Compared to DS13x271, the total time for DS14x271 should be about 10% longer and DS15x271 should be about 40% longer. Even though DS15x271 has the same number of WUs, the average time per WU will go up about 40%.
ID: 2617 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Speedy51

Send message
Joined: 13 Apr 19
Posts: 13
Credit: 2,152,705
RAC: 9,561
Message 2618 - Posted: 4 Jan 2020, 7:39:23 UTC - in response to Message 2617.  

Appreciate the update. I am assuming going off past experience DS15×271 will take about the same amount of time as DS13×271 if not longer simply because it has the same number of work units in the batch?


Compared to DS13x271, the total time for DS14x271 should be about 10% longer and DS15x271 should be about 40% longer. Even though DS15x271 has the same number of WUs, the average time per WU will go up about 40%.

Thank you for this information. When we hit these longer ones I will be using my GPU only
ID: 2618 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Eric Driver
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 8 Jul 11
Posts: 1045
Credit: 125,908,995
RAC: 110,673
Message 2632 - Posted: 11 Feb 2020, 20:37:49 UTC - in response to Message 2568.  

In case anyone notices...

I have dropped in a few WUs for the {2,7} case. There are only several thousand of these and should be relatively quick, so not worth putting on the batch status page. I usually run these extra cases offline, but these were a little larger than usual.

The final results will be some extra rows in this table (look for the entries with S={2,7}):
https://numberfields.asu.edu/NumberFields/FieldTables/Decics52.html
ID: 2632 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Eric Driver
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 8 Jul 11
Posts: 1045
Credit: 125,908,995
RAC: 110,673
Message 2701 - Posted: 29 Mar 2020, 16:05:01 UTC - in response to Message 2613.  

Just a quick status update... Data set 14x271 is almost complete (just waiting for final WUs to be returned). Moving on to Data set 15x271.
ID: 2701 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Speedy51

Send message
Joined: 13 Apr 19
Posts: 13
Credit: 2,152,705
RAC: 9,561
Message 2706 - Posted: 5 Apr 2020, 0:29:35 UTC - in response to Message 2701.  

According to that the batch status page all of Data set 14x271 has been returned
ID: 2706 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Eric Driver
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 8 Jul 11
Posts: 1045
Credit: 125,908,995
RAC: 110,673
Message 2707 - Posted: 5 Apr 2020, 4:44:59 UTC - in response to Message 2706.  

According to that the batch status page all of Data set 14x271 has been returned


Actually, there are about 2k left. I marked it as done since it was 99.95% complete.
ID: 2707 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Speedy51

Send message
Joined: 13 Apr 19
Posts: 13
Credit: 2,152,705
RAC: 9,561
Message 2708 - Posted: 5 Apr 2020, 21:19:57 UTC - in response to Message 2707.  

Thanks for the update Eric
ID: 2708 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Eric Driver
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 8 Jul 11
Posts: 1045
Credit: 125,908,995
RAC: 110,673
Message 2818 - Posted: 8 Jun 2020, 21:15:19 UTC - in response to Message 2632.  

In case anyone notices...

I have dropped in a few WUs for the {2,7} case. There are only several thousand of these and should be relatively quick, so not worth putting on the batch status page. I usually run these extra cases offline, but these were a little larger than usual.

The final results will be some extra rows in this table (look for the entries with S={2,7}):
https://numberfields.asu.edu/NumberFields/FieldTables/Decics52.html


I have added an extra "catch all" table to the batch status page. These are quick cases relative to the current monster {2,5} case. I will release them in small amounts in parallel to the current 15x271 case so as not to perturb things too much.

The goal of these cases is to fill in some more rows of this table:
https://numberfields.asu.edu/NumberFields/FieldTables/Decics52.html
ID: 2818 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 28 Oct 11
Posts: 148
Credit: 122,940,599
RAC: 70,626
Message 2820 - Posted: 8 Jun 2020, 22:05:13 UTC - in response to Message 2818.  

I've noticed that I have received a few of the sf7_DS-16x4 tasks, which match the {2,11} row in your new table with 7680 tasks in the batch.

They're coming in with a 3 day deadline, but with a runtime estimate nearly three times the baseload batch (20,000 GFLOPs instead of 7,000 GFLOPs). That's going to skew the runtime estimates, if they're really 'relatively quick' runners.
ID: 2820 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Eric Driver
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 8 Jul 11
Posts: 1045
Credit: 125,908,995
RAC: 110,673
Message 2821 - Posted: 9 Jun 2020, 0:53:38 UTC - in response to Message 2820.  

I've noticed that I have received a few of the sf7_DS-16x4 tasks, which match the {2,11} row in your new table with 7680 tasks in the batch.

They're coming in with a 3 day deadline, but with a runtime estimate nearly three times the baseload batch (20,000 GFLOPs instead of 7,000 GFLOPs). That's going to skew the runtime estimates, if they're really 'relatively quick' runners.


I gave them a higher priority so they would go out first. The accelerated retries then drops their delay bound, something I had forgotten about. I'll disable that again until these are through.

These should be about 30% faster than the 15x271 cases on average. I recently increased the fpops_est in the template file. I did this because I was noticing people getting more work than they could finish, resulting in a lot of abortions. The hope was that the server would dish out fewer work units, at least on new hosts connecting for the first time. Maybe I increased it too much.

Also, just to clarify, by "quick" I meant the batch as a whole. Many of these new cases have only a few thousand WUs as opposed to millions. Individual tasks are nominally 1 hour.
ID: 2821 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Speedy51

Send message
Joined: 13 Apr 19
Posts: 13
Credit: 2,152,705
RAC: 9,561
Message 2824 - Posted: 10 Jun 2020, 6:55:19 UTC

Thanks Eric for adding the "miscellaneous searches" table, nice to be able to see how far through a number we are (without having to work it out)
ID: 2824 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
1 · 2 · Next

Message boards : News : Batch plan


Main page · Your account · Message boards


Copyright © 2020 Arizona State University