|
1)
Message boards :
Number crunching :
I cant send finished wus
(Message 2018)
Posted 11 Apr 2018 by Richard Haselgrove Post: And we can see that WU (23257062 - task 25321670) was reported and validated by host 95038 at 2:10:51 UTC on 10 Apr 2018. We can't see the user name ('anonymous'), to match the 'hidden' setting for your computers. But you might be able to see some host log activity around that time. |
|
2)
Message boards :
Number crunching :
I cant send finished wus
(Message 2016)
Posted 11 Apr 2018 by Richard Haselgrove Post: Here is an absolutely off-the-wall observation, which may be completely unrelated - but it has some similarities with this situation. I present it as food for thought, nothing more. Like HerrJeh, I have multiple machines - all are at my home, and share an internet connection. They have the same public IP addresses, but different private IP addresses behind the router. I'm connected to a different BOINC project, which like this one runs every BOINC function from a single server. That other project distributes long-running (up to 18 hour) workunits, but regards them as time-critical - it likes to have them returned within 24 hours. So I don't want to get a cache of tasks in advance, and in fact the project often has no tasks immediately available on demand. All of which is a long way of explaining why I find myself repeatedly clicking the 'Update' button in an attempt to get new work before the current task completes. What I am observing is that I can issue repeated requests every 30 seconds from one machine, and connect to the server every time. But if I try to connect from a different machine (same LAN, same IP) in between, the second machine can't connect. If I stop updating the first machine for a minute or two, the second machine can connect, and goes on connecting for as long as is needed and allowed by their 30-second backoff interval. The nature of the server contact required doesn't affect the connection failures: I've just had a machine which couldn't connect to upload results, while I was requesting new work on a different machine. It isn't simply congestion at the server port: that would be more random. This observation is strictly about multiple connection attempts, closely spaced in time, from different computers sharing the same public IP address. It feels more like a server OS-level problem than a BOINC problem, and it's been consistent for weeks, if not months. Their server is running 11/04/2018 11:03:50 | | [http] [ID#1] Received header from server: Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips mod_auth_gssapi/1.3.1 mod_auth_kerb/5.4 mod_fcgid/2.3.9 PHP/5.4.16 mod_wsgi/3.4 Python/2.7.5My machines are all Windows 7/64 and run recent versions of BOINC (mostly v7.9 test builds). I haven't yet explored the http logs for further clues, but I'll give it a try when I have time. |
|
3)
Message boards :
Number crunching :
I cant send finished wus
(Message 2009)
Posted 10 Apr 2018 by Richard Haselgrove Post: hosts that could return Decic tasks but not Septic tasks.I noticed that in the logs, and couldn't understand it. Both types are uploading from my machines with no distinction. All communications - specifically including uploading - are handled by BOINC (the same BOINC in both cases), so there should be no difference. The build tools and libraries used in your applications should make no difference either - the comms happen after your app has already shut down. I looked at the destination of each type: <upload_url>http://numberfields.asu.edu/NumberFields_cgi/file_upload_handler/</upload_url> (septics) <upload_url>http://numberfields.asu.edu/NumberFields_cgi/file_upload_handler/</upload_url> (sf4_DS-13x271) and my eye can't spot a difference. Is there a difference in the file size offered for upload? |
|
4)
Message boards :
Number crunching :
I cant send finished wus
(Message 1999)
Posted 4 Apr 2018 by Richard Haselgrove Post: Maybe others are also unable to return or get new work?I've not noticed any problems here, across multiple Windows machines. Here's the http_debug for a successful upload, for comparison. I don't think xfer_debug would add much extra information (though no harm in selecting it): these problems usually occur during the header negotiation. 04/04/2018 09:26:18 | NumberFields@home | [http] [ID#15238] Sent header to server: Accept: */*If the problem is widespread, one possible cause is the security certificate verification you see here. I'm using a test version of BOINC (v7.9.2) which includes an updated ca-bundle.crt file: the latest test versions (currently v7.9.3, soon to be v7.9.4) are available at https://boinc.berkeley.edu/download_all.php |
|
5)
Message boards :
News :
Minor server overload problems
(Message 1973)
Posted 27 Feb 2018 by Richard Haselgrove Post: I was in a conference call this evening with a couple of *very* experienced BOINC server administrators. One thought that on a lightly-loaded project, a database connection might well time out between active requests: the other had never seen such a thing. On reflection, both thought there might be some useful information about a possible database server stoppage in the MySQL logs. They suggested I copy your question to the boinc_projects mailing list, both to remind them to look again, and to get some broader responses from the community. OK if I do that in the morning? |
|
6)
Message boards :
News :
Minor server overload problems
(Message 1971)
Posted 27 Feb 2018 by Richard Haselgrove Post: The server has gone into the 'feeder not running' maintenance mode again - 1 hour backoff on scheduler requests, no reporting and no new work. Edit - sorry, working now. Panic over. |
|
7)
Message boards :
News :
Database crash
(Message 1939)
Posted 8 Jan 2018 by Richard Haselgrove Post: Yes, things getting back to normal here too - reported completed work and got a few new ones. I'm getting some of those "permanent http error" too. They seem to be data files missing from the download storage area (HTTP 404), so of course the downloads fail for all replications of the workunit. You'll have a few jobs to re-issue when all this is over. |
|
8)
Message boards :
News :
Minor server overload problems
(Message 1929)
Posted 31 Dec 2017 by Richard Haselgrove Post: It's fairly well known that a stopped feeder puts the server into a form of maintenance mode - nothing gets done, volunteer hosts are backed off for 1 hour (as my log shows). I presume this is deliberate to stop the situation getting worse until it can be inspected. What that doesn't say is why the feeder stopped in the first place - your second post about the lack of temp file space seems as good an explanation as any. At intervals throughout the day, I've seen the project come back up fully (so I could report and refill): then go into full maintenance mode with these boards down as well: then return to normal working (the current state: just reported and received new tasks). Which sounds like a good moment to go out and start celebrating the new year... |
|
9)
Message boards :
News :
Minor server overload problems
(Message 1926)
Posted 31 Dec 2017 by Richard Haselgrove Post: Probably related to the other problems, I'm currently unable to report completed tasks: the messages I get are 31/12/2017 09:27:54 | NumberFields@home | Server error: feeder not runningNot a problem - they can sit here until you're ready for them. |
|
10)
Message boards :
Number crunching :
No work 'ready to send'
(Message 1903)
Posted 6 Nov 2017 by Richard Haselgrove Post: Yes, I'm topping up nicely. I guess the new batch ran more quickly than our machines are used to after DS12x271. |
|
11)
Message boards :
Number crunching :
No work 'ready to send'
(Message 1901)
Posted 5 Nov 2017 by Richard Haselgrove Post: Although you opened up new search DS13x270 for us last week, we seem to be getting ahead of you ;-) |
|
12)
Message boards :
News :
implementing SSL on the server
(Message 1784)
Posted 16 Nov 2016 by Richard Haselgrove Post: I tried out version 5.10.45, as it can be used portable on my USB drive, and received the message: You could try extracting the file 'ca-bundle.crt' from a newer BOINC download and replacing your old one with that. No promise that it will work, though. |
|
13)
Message boards :
News :
implementing SSL on the server
(Message 1778)
Posted 11 Oct 2016 by Richard Haselgrove Post: I'd be interested to hear if someone found a simpler way to connect using an older machine/manager. I see that Vitaly also has a Windows 7 machine attached to the project. If he finds the file 'account_numberfields.asu.edu_NumberFields.xml' in the root of the BOINC data directory on that machine, and copies it to the equivalent location in the data directory of the Ubuntu machine, it *may* attach that machine to the project. You may need to restart the BOINC client/service. I've used that method to attach a new Windows machine in the past, but I can't be sure that Linux will accept the Windows file format (CRLF line endings, instead of *nix LF only). Also, I did it before SSL came into widespread use: in fact, my own account file still has the http:// master url, and I keep getting nagged to detach and re-attach 'when convenient'. But since it's currently still working, I haven't bothered. |
|
14)
Message boards :
Number crunching :
Upload problems?
(Message 1771)
Posted 1 Sep 2016 by Richard Haselgrove Post: I had that all day (starting about 07:00 UTC): the "transient HTTP error" in question was a server timeout, according to the http_debug log. I had the same timeout error when attempting to access this website, but when this site came back to life again (about an hour and a half ago), the uploads resumed too. Worth giving them a prod with the Transfers::Retry Now button. |
|
15)
Message boards :
Number crunching :
Qsqrt421_DS3x8 15+ hour running time?
(Message 1752)
Posted 6 Jul 2016 by Richard Haselgrove Post: I reckon I've got four of those: wu_Qsqrt421_DS1x8_CV1_S1000_N2_21_N1_-6868to-4346 (the original _0 dated 14 June, still running on a slower machine) wu_Qsqrt421_DS1x8_CV1_S1000_N2_41_N1_-7063to-4131 wu_Qsqrt421_DS1x8_CV1_S1000_N2_10_N1_-4391to-1809 (I'll abort that one, someone else has completed it) wu_Qsqrt421_DS1x8_CV1_S1000_N2_28_N1_-4197to-1862 wu_Qsqrt421_DS1x8_CV1_S1000_N2_49_N1_-3971to-1938 had already slipped through - not as slow as the others - before I saw your post. |
|
16)
Message boards :
Number crunching :
Qsqrt421_DS3x8 15+ hour running time?
(Message 1749)
Posted 27 Jun 2016 by Richard Haselgrove Post: Please don't feel you have to. I had wu_Qsqrt421_DS1x8_CV1_S1000_N2_49_N1_-7142to-4045_0 for several days last week, but it finished fine and within the original deadline. There does seem to be a class of workunit which spends a lot of time on the first few %age steps at the start of computation, and a similar slow phase at the end, but which runs through the middle section very quickly. |
|
17)
Message boards :
News :
Bounded App Final Tally of Results
(Message 1731)
Posted 23 May 2016 by Richard Haselgrove Post: Yes. The original Hobbes link gave me, and is still giving, This site can’t be reached The use of the word "refused" implies something more specific than a simple failure. The locally-hosted copy displays fine, though. |
|
18)
Message boards :
Number crunching :
Upload problems May 12
(Message 1723)
Posted 12 May 2016 by Richard Haselgrove Post: LOL - posting about it did the trick. All cleared now. |
|
19)
Message boards :
Number crunching :
Upload problems May 12
(Message 1722)
Posted 12 May 2016 by Richard Haselgrove Post: I'm getting upload failures on three machines so far. 12/05/2016 08:58:23 | NumberFields@home | [http] [ID#108] Info: Connected to numberfields.asu.edu (129.219.51.76) port 80 (#191) 12/05/2016 08:58:23 | NumberFields@home | [http] [ID#108] Sent header to server: POST /NumberFields_cgi/file_upload_handler/ HTTP/1.1 12/05/2016 08:58:23 | NumberFields@home | [http] [ID#108] Sent header to server: Host: numberfields.asu.edu 12/05/2016 08:58:23 | NumberFields@home | [http] [ID#108] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.6.22) 12/05/2016 08:58:23 | NumberFields@home | [http] [ID#108] Sent header to server: Accept: */* 12/05/2016 08:58:23 | NumberFields@home | [http] [ID#108] Sent header to server: Accept-Encoding: deflate, gzip 12/05/2016 08:58:23 | NumberFields@home | [http] [ID#108] Sent header to server: Content-Type: application/x-www-form-urlencoded 12/05/2016 08:58:23 | NumberFields@home | [http] [ID#108] Sent header to server: Accept-Language: en_GB 12/05/2016 08:58:23 | NumberFields@home | [http] [ID#108] Sent header to server: Content-Length: 722 12/05/2016 08:58:23 | NumberFields@home | [http] [ID#108] Sent header to server: 12/05/2016 08:58:23 | NumberFields@home | [http] [ID#108] Info: We are completely uploaded and fine 12/05/2016 08:59:28 | NumberFields@home | [http] [ID#108] Info: Operation too slow. Less than 10 bytes/sec transferred the last 60 seconds 12/05/2016 08:59:28 | NumberFields@home | [http] [ID#108] Info: Closing connection 191 12/05/2016 08:59:28 | NumberFields@home | [http] HTTP error: Timeout was reached 12/05/2016 08:59:28 | NumberFields@home | Temporarily failed upload of wu_sf3_DS-11x271_Grp158560of614400_0_0: transient HTTP error 12/05/2016 08:59:28 | NumberFields@home | Backing off 01:51:48 on upload of wu_sf3_DS-11x271_Grp158560of614400_0_0 The upload appears to complete, but I don't get the necessary acknowledgement that all is well. |
|
20)
Message boards :
Number crunching :
Long running wu_Qsqrt421_DS1x5 units - how long to let them run?
(Message 1694)
Posted 19 Apr 2016 by Richard Haselgrove Post: This was explained to me once, many years ago - by Rom Walton, I think. The action was Exit status 203 (0xcb) EXIT_ABORTED_VIA_GUI - user clicked the abort button. They (the BOINC developers) deliberately coded it to throw a breakpoint on abort, to get some state debug information and feed it back to the developer. Perhaps, like in this thread, the user aborted it because it seemed to be in an infinite loop? If so, we'd like to know which code segment had the bug so it could be fixed. There aren't many developer aids in the standard BOINC client: why they chose this one, I don't know. Maybe there were a lot of infinite loops in their test code? |