Thoughts on Credit allocation


Advanced search

Message boards : Number crunching : Thoughts on Credit allocation

1 · 2 · Next
Author Message
Profile HerveUAE
Avatar
Send message
Joined: 18 Dec 16
Posts: 44
Credit: 509,829
RAC: 0
    
Message 5890 - Posted: 28 Dec 2016, 21:15:56 UTC

Hi,

After some days using ATLAS@Home, I was a bit puzzled by the credits allocated to each completed task. So I did some stats on several runs. I will share with you 2 findings that surprised me.

1. Credit is proportionate to the Run time

I read this somewhere as a general BOINC rule. Except for some variation over days, my stats show that this rule is followed for both single-core and multi-core ATLAS applications.

For multi-core, it is so true that it does not actually depend on the number of cores that were configured in app_config.xml. In my case, I found an average of 220 credit points per hour. To illustrate, let's assume that a typical task needs 8 hours of CPU time:
- With 8 cores WUs you get 5280 credit completing 24 tasks in the day.
- With 1 core multi-core WUs (it can be configured in app_config.xml) you also get 5280 credit completing only 3 tasks in the day.
* For simplification purpose, I exclude the time spent in starting the VM, etc.

2. Multi-core versus single-core

If you run the same 1 core multi-core WU in 8 hours, you would get 8 * 220 = 1760 credit. However if you run it as single-core, you would only get 8 * 45 = 360. Your computer has done the exact same task in the same time, but your credit is 5 times less.

Conclusion: if you are looking for maximum credit, just run as many 1 core multi-core WUs as your RAM permits, and use the rest of your cores on other projects ;)

I am sure the subject comes up regularly and I am just adding to a long list of similar comments. However I am still surprised that so little weight is given to the actual number crunching....

Herve
____________
We are the product of random evolution.

Erich
Send message
Joined: 18 Dec 15
Posts: 253
Credit: 1,942,248
RAC: 0
    
Message 5892 - Posted: 29 Dec 2016, 16:42:57 UTC - in response to Message 5890.
Last modified: 29 Dec 2016, 16:46:00 UTC


Conclusion: if you are looking for maximum credit, just run as many 1 core multi-core WUs as your RAM permits, and use the rest of your cores on other projects ;)

this, however, is strongly restricted by one fact which I learned only recently in the following posting by the user computezrmle:

On the other hand your BOINC client calculates the complete RAM requirement from the values in <rsc_memory_bound>.
This variable is sent by the server with each WU (ATLAS: 8900 MB) and can not be configured locally.

4 WUs would need 35.6 GB RAM and this exceeds your physical RAM size.
The 3x3 setup you had before requested only 26.7 GB.


which means that for every active ATLAS Multicore task, BOINC requests 8900MB RAM (although this is not being used up at all during crunching the task).

I got the above information as answer to my posted question, why only 3 ATLAS Multicore tasks are running simultaneously, although according to my app_config.xml there should be 4 tasks 2 cores each running.

For explanation: I have a 6+6(HT) core processor and 32GB RAM.

Due to this strange BOINC requirement of 8900MB RAM per ATLAS Multicore task, my system with 32GB RAM can run only 3 Multicores at a time, regardless of the number of cores assigned per task. This is bad, but obviously cannot be changed :-(

Erich
Send message
Joined: 18 Dec 15
Posts: 253
Credit: 1,942,248
RAC: 0
    
Message 5893 - Posted: 29 Dec 2016, 17:08:43 UTC - in response to Message 5890.

For multi-core, it is so true that it does not actually depend on the number of cores that were configured in app_config.xml

with my 2 PCs, the situation though is quite different:

on the system where I run 3 tasks 3 cores each simultaneously, the avarage credit for each task is 77 per hour. Processor speed 3.4GHz.
(runtime 10,052.92 secs, CPU time 27,390.14secs, credit 214.83)

on the other system where I run 1 task with 2 cores, the avarage credit for this task is 35 per hour.
(runtime 20,726.68 secs, CPU time 20,297.75 secs, credit 201.19)
The proceccor of this system runs @ 2.83GHz; on the other hand, this CPU has 4 "real" cores, not HT. Whereas the other system where I run 3x3, has 6+6(HT) cores. So the processor with 2.83GHz should even be more productive in this case.

At the bottom line: at least from what my experience shows, I could not agree to the statement quoted on top of this posting.

Profile HerveUAE
Avatar
Send message
Joined: 18 Dec 16
Posts: 44
Credit: 509,829
RAC: 0
    
Message 5900 - Posted: 30 Dec 2016, 8:48:36 UTC - in response to Message 5893.

Hi Erich,

I do agree with both of your comments.

At the bottom line: at least from what my experience shows, I could not agree to the statement quoted on top of this posting.

I also have 2 computers, one with 4 cores and onr with 8 cores. Here are more details of the stats I have seen:
** Credits per hour for single-core tasks:
    Intel i7-4510U 2.00GHz 4 core: 25
    Intel i7-4770K 3.50GHz 8 core: 45

** Credits per hour for multi-core tasks:

    Intel i7-4510U 2.00GHz 4 core 8 Gbytes RAM: 120 (server assumes 4 cores)
    Intel i7-4770K 3.50GHz 8 core 4 Gbytes RAM: 30 (server assumes 1 core)
    Intel i7-4770K 3.50GHz 8 core 8 Gbytes RAM: 160 (server assumes 6 cores)
    Intel i7-4770K 3.50GHz 8 core 16 Gbytes RAM: 220 (server assumes 8 cores)
    Intel i7-4770K 3.50GHz 8 core 32 Gbytes RAM: 220 (server assumes 8 cores)

So yes, the credit allocation depends on the physical characteristics of your computer.

which means that for every active ATLAS Multicore task, BOINC requests 8900MB RAM (although this is not being used up at all during crunching the task).

The BOINC scheduler considers only the RAM requirements as set by the server. Is the server assumes 4 cores, the scheduler will wait for 5700 Mbytes of RAM to be free, even of you actually use only 4100 because you configure your multi-core to use only 2 cores.

Strangely enough though, the scheduler do take into account the number of cores you have locally configured, but not of the quantity of RAM. It does not sound consistent. Bug?

Erich
Send message
Joined: 18 Dec 15
Posts: 253
Credit: 1,942,248
RAC: 0
    
Message 5901 - Posted: 30 Dec 2016, 10:16:49 UTC

Hi Herve,

I agree that not everything in the BOINC schedule sounds consistent. I have been playing around quite a lot with different ATLAS settings and was wondering often enough about the results :-)

For the one system with the 32 GB RAM (CPU is: i7-4930k @ 3.4GHz) the best configuration seems to be 3 tasks 3 cores ea.
So I am using 9 out of the 12 available cores, 2 are being used by GPUGRID. One core I have left unused; so the maximum CPU use (as shown in the Windows task manager) is about 93-95%.
A total of about 17-18 GB RAM is being used for these 3 tasks à 3 cores.

Regarding the credits, I end up with about the same as you. As said above, each of the 3-core tasks yields 77 points per hour, so if I have 3 tasks à 3 cores running, I have 231 points per hour.
You show 220 points for 8 cores (with 16 or 32 GB RAM). So this is somewhat in line, I guess.

Profile Yeti
Avatar
Send message
Joined: 20 Jul 14
Posts: 699
Credit: 22,597,832
RAC: 0
    
Message 5902 - Posted: 30 Dec 2016, 12:57:36 UTC - in response to Message 5890.

Conclusion: if you are looking for maximum credit, just run as many 1 core multi-core WUs as your RAM permits, and use the rest of your cores on other projects ;)

Nope, this is not what will really work.

Before we got MultiCore-WUs, my AverageCredit was something round about 25.000 credits per day.

With MultiCore-WUs, I could raise the AverageCredit per day to something 35.000 Credits. (I can use a little bit more cores running Atlas with MultiCore-WUs, as there is less overhead)

Atlas is not the best choice if you want to get the highest possible credits out of your available Hardware; other projects are more generous with credits.

But for me, Atlas is the most interesting project of all available BOINC-Projects

Profile HerveUAE
Avatar
Send message
Joined: 18 Dec 16
Posts: 44
Credit: 509,829
RAC: 0
    
Message 5904 - Posted: 30 Dec 2016, 19:07:35 UTC - in response to Message 5902.

other projects are more generous with credits

My initial comment was really meant about the way ATLAS@Home is calculating credit.
Local adjustments with the app_config.xml file should somehow be taken into account in the credit calculation, I think. But it may not be technically feasible.

Atlas is the most interesting project of all available BOINC-Projects

I initially joined after reading an interview with someone working at the LHC. Since then, I looked at other BOINC projects and just joined Einstein@Home which I also find interesting.
The application names from Einstein@Home give some hint at the kind of physical phenomena your computer is crunching at.
For ATLAS@Home, it's less clear to me at the moment. What is behind "Task mc15_13TeV hdamp517p5_nonallhad.simul" for example? It would be great to have some kind of short description of the task. But that is a different topic altogether.
____________
We are the product of random evolution.

Erich
Send message
Joined: 18 Dec 15
Posts: 253
Credit: 1,942,248
RAC: 0
    
Message 5911 - Posted: 1 Jan 2017, 19:10:31 UTC - in response to Message 5900.

Intel i7-4510U 2.00GHz 4 core 8 Gbytes RAM: 120 (server assumes 4 cores)
Intel i7-4770K 3.50GHz 8 core 4 Gbytes RAM: 30 (server assumes 1 core)
Intel i7-4770K 3.50GHz 8 core 8 Gbytes RAM: 160 (server assumes 6 cores)
Intel i7-4770K 3.50GHz 8 core 16 Gbytes RAM: 220 (server assumes 8 cores)
Intel i7-4770K 3.50GHz 8 core 32 Gbytes RAM: 220 (server assumes 8 cores)


Herve, how did you manage to get ATLAS running with only 4 GB RAM?

Today, I tried to attach ATLAS to one of my notebooks with 4 GB RAM, but neither 1x1 multi-core task nor 1 single-core will run - in the BOINC status column, it always says "waiting for memory".
About 2,5 GB RAM out of the 4 GB are free. Still it doesn't work :-(

Profile HerveUAE
Avatar
Send message
Joined: 18 Dec 16
Posts: 44
Credit: 509,829
RAC: 0
    
Message 5913 - Posted: 2 Jan 2017, 9:23:53 UTC - in response to Message 5911.

Erich, I answered in your other thread: mainly by playing with the BOINC Manager settings.

Erich
Send message
Joined: 18 Dec 15
Posts: 253
Credit: 1,942,248
RAC: 0
    
Message 5919 - Posted: 2 Jan 2017, 17:22:53 UTC

As said in my other thread: I got be problem solved by switching on the virtualization in the BIOS (I had not expected that this is switched off).
So, ATLAS is not running fine, so far I the single-core tasks.

I might switch to multicore (1 task with 1 core), but I am still wondering whether this would work with 4 GB RAM, given this 8.9GB RAM requirement for Multicore tasks.

Herve, did this configuration

Intel i7-4770K 3.50GHz 8 core 4 Gbytes RAM: (server assumes 1 core)

really work with multicore?

Profile Yeti
Avatar
Send message
Joined: 20 Jul 14
Posts: 699
Credit: 22,597,832
RAC: 0
    
Message 5920 - Posted: 2 Jan 2017, 17:28:48 UTC

If you want to run Multicore on a single Core, this is possible with 4 GB.

Don't use an app_config, but limit BOINC to use only 25% of available cores ( This will result on an 4 Core-PC in an 1-Core-MultiCoreWU )

Erich
Send message
Joined: 18 Dec 15
Posts: 253
Credit: 1,942,248
RAC: 0
    
Message 5921 - Posted: 2 Jan 2017, 17:50:03 UTC - in response to Message 5920.


Don't use an app_config, but limit BOINC to use only 25% of available cores ( This will result on an 4 Core-PC in an 1-Core-MultiCoreWU )

Thanks, Yeti, for the hint.
This is an interesting approach. Why would you perfer this solution over the app_config? I am asking just out of curiosity.

Profile Yeti
Avatar
Send message
Joined: 20 Jul 14
Posts: 699
Credit: 22,597,832
RAC: 0
    
Message 5922 - Posted: 2 Jan 2017, 17:55:59 UTC - in response to Message 5921.

This is an interesting approach. Why would you perfer this solution over the app_config? I am asking just out of curiosity.


An app_config overrides the correct memory-calculation of the server ...

Buy the way, 1 MultiCoreWU with 1 Core will need ( 2,5 GB + 0,8 * 1 Cores = ) 3,3 GB

Maybe this is too much for 4 GB ...

Erich
Send message
Joined: 18 Dec 15
Posts: 253
Credit: 1,942,248
RAC: 0
    
Message 5923 - Posted: 2 Jan 2017, 19:00:31 UTC - in response to Message 5922.


Buy the way, 1 MultiCoreWU with 1 Core will need ( 2,5 GB + 0,8 * 1 Cores = ) 3,3 GB

Maybe this is too much for 4 GB ...

definitely so. Without running ATLAS, the free RAM is around 2,5GB.
Now, with 1 single-core running, free RAM is between 620 and 690MB.

So, since 1 Multicore needs 3,3GB, I will refrain from even trying Multicore.

Profile HerveUAE
Avatar
Send message
Joined: 18 Dec 16
Posts: 44
Credit: 509,829
RAC: 0
    
Message 5924 - Posted: 3 Jan 2017, 2:52:14 UTC - in response to Message 5923.
Last modified: 3 Jan 2017, 2:57:24 UTC

The 8900 Mbytes that you mention Erich is only if the server considers that your computer can support 8-core WUs. With 4 Gbytes, the server will consider that you support only 1-core WUs and, as mentioned by Yeti, BOINC Manager and the VM automatically look for 3300 Mbytes without setting it in app_config.xml.

When I tested it, the computer remained responsive and usable. But my swap is on SSD, which surely helped.

Erich
Send message
Joined: 18 Dec 15
Posts: 253
Credit: 1,942,248
RAC: 0
    
Message 5925 - Posted: 3 Jan 2017, 10:17:58 UTC - in response to Message 5922.
Last modified: 3 Jan 2017, 10:21:27 UTC

Buy the way, 1 MultiCoreWU with 1 Core will need ( 2,5 GB + 0,8 * 1 Cores = ) 3,3 GB
Maybe this is too much for 4 GB ...

One final question on this:
Will the crunching of such a 1-core multicore task indeed use up to 3,3 GB RAM, or is this value just some kind of check by the program, before crunching starts?

The reason why I am asking this is: on my other PC (32GB RAM), on which I am running 3 multicore tasks 3 cores each, a maximum of 18,5GB RAM is being used for ATLAS. So each core is using slightly more than 2GB RAM.

Erich
Send message
Joined: 18 Dec 15
Posts: 253
Credit: 1,942,248
RAC: 0
    
Message 5926 - Posted: 3 Jan 2017, 11:33:25 UTC - in response to Message 5925.
Last modified: 3 Jan 2017, 11:37:43 UTC

The reason why I am asking this is: on my other PC (32GB RAM), on which I am running 3 multicore tasks 3 cores each, a maximum of 18,5GB RAM is being used for ATLAS. So each core is using slightly more than 2GB RAM.

Interesting comparison though: on the other PC with 8 GB RAM where I have one 2-core task running, ATLAS uses around 5,5 GB RAM.
Which is a marked higher per-core-value than on the PC with the 3x3 configuration.

Profile Yeti
Avatar
Send message
Joined: 20 Jul 14
Posts: 699
Credit: 22,597,832
RAC: 0
    
Message 5927 - Posted: 3 Jan 2017, 11:48:35 UTC - in response to Message 5926.

Interesting comparison though: on the other PC with 8 GB RAM where I have one 2-core task running, ATLAS uses around 5,5 GB RAM.
Which is a marked higher per-core-value than on the PC with the 3x3 configuration.

Sorry, but NO, this is not interesting or magic or whatever

This is simply the result of this non linear Atlas-Formula for memory needs:

2,5 GB + (NumberOfCores) * 0,8

This is the formula, Atlas-Team has published for their memory-calculation of MultiCoreWUs

Erich
Send message
Joined: 18 Dec 15
Posts: 253
Credit: 1,942,248
RAC: 0
    
Message 5928 - Posted: 3 Jan 2017, 12:03:44 UTC
Last modified: 3 Jan 2017, 12:32:05 UTC

okay, but how come that my PC with the 1x2-core task then needs around 5,5 GB RAM for ATLAS?
According to the formula, this should be 4.1 GB.

Profile Yeti
Avatar
Send message
Joined: 20 Jul 14
Posts: 699
Credit: 22,597,832
RAC: 0
    
Message 5929 - Posted: 3 Jan 2017, 13:20:26 UTC - in response to Message 5928.

okay, but how come that my PC with the 1x2-core task then needs around 5,5 GB RAM for ATLAS?
According to the formula, this should be 4.1 GB.

HM, where do you take the 5,5 GB RAM for Atlas from ?

There are several possible reasons:

unusual WU
memory leak
the way the total amount is taken

1 · 2 · Next

Message boards : Number crunching : Thoughts on Credit allocation