Super long estimated times

Message boards : Number crunching : Super long estimated times
Message board moderation

To post messages, you must log in.

Previous · 1 · 2

AuthorMessage
Profile Eric Driver
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 8 Jul 11
Posts: 1323
Credit: 411,786,290
RAC: 251,415
Message 1235 - Posted: 27 Feb 2015, 16:29:31 UTC - in response to Message 1234.  

Something similar going on in several of my Get Decics with Bounded Discriminant v3.00 WU's. The last one was showing 10:36:09 worked - not bad in itself, but only 26.233% complete, and that represented just an 0.1% increment in completion over its last 26 CPU minutes (i7-3770 @3.4GHz). ETC inflated by 63 minutes in the same time period. I normally wouldn't mind letting a WU crank for as long as it takes (I also participate in CPDN). This one I thought it best to abort because to keep it around was projecting a risk to other WUs that were waiting in the queue. Cumulative result of two other v3.00 WUs running to completion in the 25-30 CPU hr range in the last week - although some are much shorter - coupled with a lot of Decic Fields 1.02 WUs running 10-25 hrs lately. Not sure what can be done; maybe increase the due date for all WUs by some further percent until the tasks return to a more moderate part of the problem space.

Thanks for reporting. The current deadline is 7 days with a 3 day grace period, giving a total of 10 days. So I could raise it a little bit, but I wouldn't want to go much beyond 10 days.

When I find some time I will analyze the data from the latest subfield (which recently completed). Several weeks ago I did a spot check of the long running results- the vast majority were from the windows version of the app. Of course this could be because the vast majority of users use windows, but it's something else to look into.

This sounds like a good moment to mention WU 9446590.

The first two copies both exceeded the deadline, and I have the third. It's been running slowly but steadily for 114 hours so far, and has reached 37.317%. I get the impression that it sometimes moves on by 1% or 2% quite quickly, but usually progress moves on in 0.001% increments. Other tasks have come and gone on the other CPU cores while this one has been running, so I don't think it's an issue with the computer it's running on.

The Martinet search report for the task which has finished shows 23 cases, from "a5 = -8 + 6w" to "a5 = 14 + 6w". Mine has reached case 0, which seems about right for 37.319% - it's moved on while I've been typing!

I'll keep it running to see what happens, although I'm expecting it will far exceed the local deadline tomorrow night, and also exceed the grace period on 3 March.


Thanks for pointing this one out. It sounds like a good candidate for me to do some testing on. Since it was eventually returned and validated (past the deadline), you don't need to waste the cpu cycles, unless you really want to help me test it out.
ID: 1235 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 28 Oct 11
Posts: 179
Credit: 224,217,062
RAC: 113,131
Message 1236 - Posted: 27 Feb 2015, 18:43:27 UTC - in response to Message 1235.  

Thanks for pointing this one out. It sounds like a good candidate for me to do some testing on. Since it was eventually returned and validated (past the deadline), you don't need to waste the cpu cycles, unless you really want to help me test it out.

Happy to let it run, and report any tests you'd like me to try. Currently at 41.653% after 123 hours 34 mins.
ID: 1236 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 28 Oct 11
Posts: 179
Credit: 224,217,062
RAC: 113,131
Message 1237 - Posted: 27 Feb 2015, 22:51:33 UTC - in response to Message 1236.  

Thanks for pointing this one out. It sounds like a good candidate for me to do some testing on. Since it was eventually returned and validated (past the deadline), you don't need to waste the cpu cycles, unless you really want to help me test it out.

Happy to let it run, and report any tests you'd like me to try. Currently at 41.653% after 123 hours 34 mins.

Went out for dinner, came back. Currently at 41.757% - definitely variable speed.
ID: 1237 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Eric Driver
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 8 Jul 11
Posts: 1323
Credit: 411,786,290
RAC: 251,415
Message 1238 - Posted: 28 Feb 2015, 17:26:03 UTC - in response to Message 1237.  

Thanks for pointing this one out. It sounds like a good candidate for me to do some testing on. Since it was eventually returned and validated (past the deadline), you don't need to waste the cpu cycles, unless you really want to help me test it out.

Happy to let it run, and report any tests you'd like me to try. Currently at 41.653% after 123 hours 34 mins.

Went out for dinner, came back. Currently at 41.757% - definitely variable speed.


Thanks for doing that. As a test, I am running it on my linux box (3.6 GHz core i7). It looks like it is almost finished and it's been about 24 hours. Longer than I'd like, but still much better than what you guys are reporting. So this seems to point to an issue with the windows version of the app. One big difference is that the windows version is 32 bits. I will continue looking into this. In the meantime, I will be curious to know if/when this ever finishes for you.
ID: 1238 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 28 Oct 11
Posts: 179
Credit: 224,217,062
RAC: 113,131
Message 1239 - Posted: 28 Feb 2015, 19:12:17 UTC - in response to Message 1238.  

Thanks for pointing this one out. It sounds like a good candidate for me to do some testing on. Since it was eventually returned and validated (past the deadline), you don't need to waste the cpu cycles, unless you really want to help me test it out.

Happy to let it run, and report any tests you'd like me to try. Currently at 41.653% after 123 hours 34 mins.

Went out for dinner, came back. Currently at 41.757% - definitely variable speed.

Thanks for doing that. As a test, I am running it on my linux box (3.6 GHz core i7). It looks like it is almost finished and it's been about 24 hours. Longer than I'd like, but still much better than what you guys are reporting. So this seems to point to an issue with the windows version of the app. One big difference is that the windows version is 32 bits. I will continue looking into this. In the meantime, I will be curious to know if/when this ever finishes for you.

It's still running, despite having to take a couple of breaks - one for a Windows BSOD (unrelated), and one for an recommended Windows update which I thought wouldn't need a reboot, but did. (Why does Windows Update need 5.2 MB, and a reboot, to change the Lithuanian currency symbol to € - when the manual instructions involve changing just that one byte? Or is there something they're not telling us?)

Anyway, the task has reached 46.508% after 141 hours 45 minutes.
ID: 1239 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Eric Driver
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 8 Jul 11
Posts: 1323
Credit: 411,786,290
RAC: 251,415
Message 1240 - Posted: 1 Mar 2015, 2:28:53 UTC - in response to Message 1238.  

Thanks for pointing this one out. It sounds like a good candidate for me to do some testing on. Since it was eventually returned and validated (past the deadline), you don't need to waste the cpu cycles, unless you really want to help me test it out.

Happy to let it run, and report any tests you'd like me to try. Currently at 41.653% after 123 hours 34 mins.

Went out for dinner, came back. Currently at 41.757% - definitely variable speed.


Thanks for doing that. As a test, I am running it on my linux box (3.6 GHz core i7). It looks like it is almost finished and it's been about 24 hours. Longer than I'd like, but still much better than what you guys are reporting. So this seems to point to an issue with the windows version of the app. One big difference is that the windows version is 32 bits. I will continue looking into this. In the meantime, I will be curious to know if/when this ever finishes for you.


In case you're interested, the 64 bit linux version finished after 32 hours.
ID: 1240 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 28 Oct 11
Posts: 179
Credit: 224,217,062
RAC: 113,131
Message 1241 - Posted: 3 Mar 2015, 23:25:24 UTC - in response to Message 1240.  

My long one passed even the extended 'grace period' deadline a couple of hours ago, but I'm stubborn - "I've started, so I'll finish".

Currently at 75.933% progress after 215 hours.
ID: 1241 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Eric Driver
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 8 Jul 11
Posts: 1323
Credit: 411,786,290
RAC: 251,415
Message 1242 - Posted: 4 Mar 2015, 5:00:36 UTC - in response to Message 1241.  

My long one passed even the extended 'grace period' deadline a couple of hours ago, but I'm stubborn - "I've started, so I'll finish".

Currently at 75.933% progress after 215 hours.


Good grief. I won't blame you if you decide to kill it.
ID: 1242 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 28 Oct 11
Posts: 179
Credit: 224,217,062
RAC: 113,131
Message 1243 - Posted: 4 Mar 2015, 21:16:09 UTC

Found another long one for your collection.

wu_12E10_SF76-0_Idx6_Grp45514of116450
ID: 1243 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 28 Oct 11
Posts: 179
Credit: 224,217,062
RAC: 113,131
Message 1257 - Posted: 9 Mar 2015, 0:27:04 UTC

Well, my really long one finally finished - after 336 hours. Task 10321335.

Too late to validate, of course, but I was expecting that - it's the challenge, the taking part, that counts.
ID: 1257 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Eric Driver
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 8 Jul 11
Posts: 1323
Credit: 411,786,290
RAC: 251,415
Message 1258 - Posted: 9 Mar 2015, 0:53:51 UTC - in response to Message 1257.  

Well, my really long one finally finished - after 336 hours. Task 10321335.

Too late to validate, of course, but I was expecting that - it's the challenge, the taking part, that counts.


Thanks Richard! I'm running the newest windows version of the app and it's currently at about the halfway point after 48 hours. Faster than yours but still much slower than the linux version. So I guess the latest pari code didn't fix the problem.
ID: 1258 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Rasputin42

Send message
Joined: 5 Nov 11
Posts: 25
Credit: 1,450,603
RAC: 0
Message 1261 - Posted: 4 Apr 2015, 10:41:17 UTC

I have a wu, that makes less and less progress, the longer it runns.
At36% it has already taken 45h and it is getting slower and slower.Should i abort it?
ID: 1261 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Eric Driver
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 8 Jul 11
Posts: 1323
Credit: 411,786,290
RAC: 251,415
Message 1262 - Posted: 4 Apr 2015, 18:06:28 UTC - in response to Message 1261.  

I have a wu, that makes less and less progress, the longer it runns.
At36% it has already taken 45h and it is getting slower and slower.Should i abort it?


That is not uncommon. As long as you're not past the deadline I would let it continue (remember there is a 3 day grace period after the deadline).

I have also seen some WUs go fast, then slow down for a while, and then go fast again. This seemingly random behavior has been explained before in other threads, but it doesn't hurt to explain it again via an example. Let's say you have 3 outer loop iterations. If the first iteration is fast you will see the progress bar go from 0 to 33.3% relatively quickly. If the next iteration is really slow, the progress bar will then take much longer to get to 66.6%. If the final iteration is fast, the progress meter will speed up again.

Anyways, it's hard to predict which loop iterations will be fast and which will be slow. If I could predict it, then we probably wouldn't be having this conversation.
ID: 1262 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Eric Driver
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 8 Jul 11
Posts: 1323
Credit: 411,786,290
RAC: 251,415
Message 1266 - Posted: 14 Apr 2015, 18:36:00 UTC - in response to Message 1257.  

Well, my really long one finally finished - after 336 hours. Task 10321335.

Too late to validate, of course, but I was expecting that - it's the challenge, the taking part, that counts.


I've been meaning to follow up on this thread.

First, to summarize: the 64bit linux app is about 4 times faster than the corresponding 32bit windows app. The 64bit linux app is twice as fast as the 32bit linux app. These times hold for the longer running cases (>5 hours). For cases that are less than 30 minutes, there is little improvement in the time. I believe the longer running cases are spending much more time in the deeper recesses of the algorithm where it's factoring very large integers, and this is where the 64bit version will outshine the 32bit version (I will need to use a profiler to prove this).

So it's apparent that a 64 bit version of windows would be helpful, at least for the longer running cases. To this end I've spent the last 2 weekends modifying the Pari library so that it would build with 64bit mingw. I now have a 64bit windows app that is giving the correct answers. I am currently running some timing tests. If these tests show the 2x improvement that I am hoping for, then I will promote this new app to the project.
ID: 1266 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 28 Oct 11
Posts: 179
Credit: 224,217,062
RAC: 113,131
Message 1269 - Posted: 30 Apr 2015, 7:59:32 UTC - in response to Message 1266.  

You might want to have a look at task 10714272.

Still the current version, but Windows 7/64 - stderr says

*** bug in PARI/GP (Segmentation Fault), please report.
*** Error in the PARI system. End of program.

So I have :)
ID: 1269 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Eric Driver
Project administrator
Project developer
Project tester
Project scientist

Send message
Joined: 8 Jul 11
Posts: 1323
Credit: 411,786,290
RAC: 251,415
Message 1270 - Posted: 30 Apr 2015, 18:38:22 UTC - in response to Message 1269.  

You might want to have a look at task 10714272.

Still the current version, but Windows 7/64 - stderr says

*** bug in PARI/GP (Segmentation Fault), please report.
*** Error in the PARI system. End of program.

So I have :)


Thanks. I think that's an old bug that seems to resurface everytime I upgrade Pari. The bug I'm thinking of was supposed to have been fixed in the latest version, so maybe this is a different one. I will have to look into it.


Just an fyi...I'm still testing my newer 64bit version of Pari. The project apps work fine, but when I ran the full suite of pari tests it failed in a few spots so I'm trying to hunt down the causes of that, just in case it might impact the project apps somehow.
ID: 1270 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Previous · 1 · 2

Message boards : Number crunching : Super long estimated times


Main page · Your account · Message boards


Copyright © 2024 Arizona State University