continuous drop of average credit since last week's WU problems


Advanced search

Message boards : Number crunching : continuous drop of average credit since last week's WU problems

1 · 2 · Next
Author Message
Erich
Send message
Joined: 18 Dec 15
Posts: 253
Credit: 1,942,248
RAC: 0
    
Message 5532 - Posted: 21 Oct 2016, 16:24:35 UTC
Last modified: 21 Oct 2016, 16:49:22 UTC

As you may remember, there was the major problem with validation errors on October 14 and 15 - as discussed here:
http://atlasathome.cern.ch/forum_thread.php?id=598

After the problem got solved, I resumed crunching on both PCs: one with two single-core tasks, the other with 3 multicore (3 cores ea.) tasks.
Everything, i.e. all settings etc. are exactly same as before. No changes in the hardware or software environment of the 2 PCs.

However, what I notice since then is that the figure of my "average credit" drops from day to day. Before, it was above 7.000 (rank 26), now it's at 5.892 (rank 36).
What appears to me is that now it takes each of the tasks longer to complete.
Any idea what the problem could be, and how I could get it solved?

Erich
Send message
Joined: 18 Dec 15
Posts: 253
Credit: 1,942,248
RAC: 0
    
Message 5533 - Posted: 21 Oct 2016, 20:08:04 UTC

Just to illustrate the problem in real figures.

I compared runtimes (total as well as CPU) from October 13 and the time after new WUs were available from October 15 on:

before: Runtime 18460secs / CPU time 52022secs - 354 points
after: Runtime 19377secs / CPU time 54788secs - 276 points

based on the crunching speed and resulting points of "before", there should be 371 points instead of 276.
The above example is just one of several. They all look alike.
So, no surprise that my average points figure is dropping and dropping :-(
You understand what I want to say?

PHILIPPE
Send message
Joined: 24 Jul 16
Posts: 84
Credit: 53,413
RAC: 0
    
Message 5541 - Posted: 23 Oct 2016, 9:18:29 UTC - in response to Message 5533.

I think it over your trouble differently :

I try to compare your results with the ones of georg stoil lastly and i find very curious thing in his results but i don't want to create problems, it's not my purpose.

When you look at this task:
nearly the same cpu time :49 471,61 sec , he won 739,47 credits , in fact the double of you.
Yes the double !!!
When i look inside this task , i find it ran with 4 cores WU:

Stderr output

<core_client_version>7.6.22</core_client_version>
<![CDATA[
<stderr_txt>
2016-10-17 19:13:15 (5712): vboxwrapper (7.7.26196): starting
2016-10-17 19:13:15 (5712): Feature: Checkpoint interval offset (599 seconds)
2016-10-17 19:13:15 (5712): Detected: VirtualBox COM Interface (Version: 5.1.4)
2016-10-17 19:13:15 (5712): Detected: Minimum checkpoint interval (900.000000 seconds)
2016-10-17 19:13:15 (5712): Successfully copied 'init_data.xml' to the shared directory.
2016-10-17 19:13:15 (5712): Create VM. (boinc_3e02007b3d4a5923, slot#4)
2016-10-17 19:13:15 (5712): Setting Memory Size for VM. (5700MB)
2016-10-17 19:13:15 (5712): Setting CPU Count for VM. (4)


,whereas on the table of tasks , it says 8 cores WU.

7439681 5745206 17 Oct 2016, 14:23:14 UTC 18 Oct 2016, 0:19:22 UTC Terminé et validé 14,737.87 49,471.61 737.49 8 ATLAS Simulation Running on Multiple Core v1.04 (vbox_64_mt_mcore)
windows_x86_64
[/quote]

Is there a bug ? or is this the reason why you are lower in the ranking ?

Erich
Send message
Joined: 18 Dec 15
Posts: 253
Credit: 1,942,248
RAC: 0
    
Message 5544 - Posted: 23 Oct 2016, 20:04:02 UTC - in response to Message 5541.
Last modified: 23 Oct 2016, 20:08:24 UTC

... Is there a bug ? or is this the reason why you are lower in the ranking ?

interesting point. Would be interesting to find out what's going on.

In another thread here in the forum, it was stated that with multicore operation, 3 tasks per core is a pretty efficient setting. Thats why I had changed from 9 single cores WU to 3 x 3 multicore.
As said before, at the beginning I had the impression that this change was a remarkable improvement. But then this changed.

PHILIPPE
Send message
Joined: 24 Jul 16
Posts: 84
Credit: 53,413
RAC: 0
    
Message 5545 - Posted: 24 Oct 2016, 15:52:55 UTC - in response to Message 5544.

Normally , i think the credits earned depends on the benchmark power realized at the beginning of the computation.
So It's possible for a same time cpu , 2 crunchers don't earn the same amount of credits.
I think the benchmark calculates the number of operations by seconde to estimate its real power.
But , here when we look at the computers stats :
Georg :3869.29 million d'opérations/sec
Erich :3674.75 million d'opérations/sec
I don't see the double of power.
Perhaps , the benchmark takes in consideration other parameters...(if someone knows).
-------------------------------------------------------------------------------
In the thread of Weinjing Wu, it reveals for me : this is the 4 cores the most adapted to calculate the events.

Only 221 sec by events.You should better use this option, you will treat events more quickly.You have enough memory to support the load.Your RAC (recent activity credit)should increase.This is the georg 's choice...(with Linux).
-----------------------------------------------------------------------------
But you have to think that your hardware can decrease its power with the time (clear the fans,remove unuseful services ,or reinstall windows).

maeax
Send message
Joined: 25 Jun 14
Posts: 50
Credit: 1,700,662
RAC: 0
    
Message 5546 - Posted: 24 Oct 2016, 16:45:02 UTC
Last modified: 24 Oct 2016, 16:46:12 UTC

Computer-ID 22.059 had today Multi-Core Tasks with 4 CPU's and 170.000 GFlops work
and Single-Core Tasks with 170.000 GFlops work.
Both Tasks have about 280 Cobblestones and nearby the same CPU-Time (45.000sec.).

PHILIPPE
Send message
Joined: 24 Jul 16
Posts: 84
Credit: 53,413
RAC: 0
    
Message 5549 - Posted: 24 Oct 2016, 20:58:13 UTC - in response to Message 5546.

In this case , you're right , it seems no difference between these two sorts of Work Unit, as if the benchmark power test was the same for both of the single core and four-cores.It's curious ,if the weinjing's table is right, the benchmark should be different and adjusted following the type of WUs employed.
If a same event is treated more quickly by a process (4 cores WU),than an other (1 core WU),Why the cpu time is the same ?
Are the weinjing's tests, out of date ?
----------------------------------------------------------------------------------------------------------------------------------------------------------------
Is the Cobblestone stable in the time ?
When i look computer : 45207 (same computer,same WU,1 day older)

7467329 5763321 21 Oct 2016, 3:18:35 UTC 21 Oct 2016, 8:47:21 UTC Terminé et validé 5,017.49 17,957.70 187.11 4 ATLAS Simulation Running on Multiple Core v1.04 (vbox_64_mt_mcore)
x86_64-pc-linux-gnu




7476155 5769088 22 Oct 2016, 7:25:32 UTC 22 Oct 2016, 12:59:26 UTC Terminé et validé 5,081.31 17,820.32 347.98 4 ATLAS Simulation Running on Multiple Core v1.04 (vbox_64_mt_mcore)
x86_64-pc-linux-gnu


nearly same cpu time : but big change 187.11-->347.98

The rules must be clear for every one if they have to be accepted by most of us.

----------------------------------------------------------------------------------------------------------------------------------------------------------------
Personnaly i'm not interested in credit race.Help the research (solidarity with our little means )is more valuable, but i encourage the competition for the ones who want it...(untill it stays a game).

PHILIPPE
Send message
Joined: 24 Jul 16
Posts: 84
Credit: 53,413
RAC: 0
    
Message 5566 - Posted: 31 Oct 2016, 10:55:47 UTC - in response to Message 5549.

For the ones, who need more details on the credit system , there is a full page , filled with explanations and answers (history of system , Cobblestone , CreditNew ,...).
It becomes easy to understand the difficulty of the way the credits are calculated for a given task.Many factors play for the results.

Each day is a learning.

Erich
Send message
Joined: 18 Dec 15
Posts: 253
Credit: 1,942,248
RAC: 0
    
Message 5574 - Posted: 2 Nov 2016, 11:24:29 UTC - in response to Message 5566.

For the ones, who need more details on the credit system , there is a full page , filled with explanations ...

OMG, not complicated at all :-)

Erich
Send message
Joined: 18 Dec 15
Posts: 253
Credit: 1,942,248
RAC: 0
    
Message 5584 - Posted: 7 Nov 2016, 17:17:26 UTC - in response to Message 5532.

...
However, what I notice since then is that the figure of my "average credit" drops from day to day. Before, it was above 7.000 (rank 26), now it's at 5.892 (rank 36).
What appears to me is that now it takes each of the tasks longer to complete.
Any idea what the problem could be, and how I could get it solved?


The above posting I wrote on October 21st.
Some time later, I saw a marked improvement, although I did not change anything - neither with my hardware nor with my software.
Anyway, it was a nice development.

However, since a few days, again I notice quite a continuous deterioration, i.e. getting less and less points for the same amount of runtime and CPU time.
Here the examples:

Date / runtime / CPU time / points

Nov.1: 19778 secs / 56500 secs / 473 points
Nov.5: 19736 secs / 56976 secs / 411 points
Nov.7: 19909 secs / 56485 secs / 307 points (!!!) - 35% less than 5 days ago

I am running 3 multicore tasks 3 cores ea.

Any explanation for this?

FYI: I made no changes whatsoever, hardware and software always the same.

Erich
Send message
Joined: 18 Dec 15
Posts: 253
Credit: 1,942,248
RAC: 0
    
Message 5685 - Posted: 19 Nov 2016, 6:41:37 UTC - in response to Message 5584.
Last modified: 19 Nov 2016, 6:41:58 UTC

Another strange case, from a PC different to the one from which the above sample cames:

Date / task number / runtime / CPU time / points

Nov. 17: 7664980 / 39399 secs / 38878 secs / 1064 points
Nov. 18: 7667999 / 38800 secs / 39290 secs / 613 points

Same settings, no changes whatsoever in hardware or software between Nov. 17 and Nov. 18, but 42% (!) less points.

Maybe I don't understand the system how the points are being assigned. Or anything is indeed wrong with the way the points are assigned - no idea.
Can anyone enlighten me?

Erich
Send message
Joined: 18 Dec 15
Posts: 253
Credit: 1,942,248
RAC: 0
    
Message 5687 - Posted: 19 Nov 2016, 16:08:45 UTC - in response to Message 5685.

Nov. 18: 7667999 / 38800 secs / 39290 secs / 613 points

sorry, there was a typo in the above line.
runtime was 39800 secs

PHILIPPE
Send message
Joined: 24 Jul 16
Posts: 84
Credit: 53,413
RAC: 0
    
Message 5688 - Posted: 19 Nov 2016, 18:30:34 UTC - in response to Message 5687.

Perhaps , Erich , the credit system keeps in memory the fact several previous Wu failed in the past.
So , with the purpose of reducing the loss of credits you have because of bad Wus(endless wu aborted or with errors which are not of your responsability,for instance this one) , the credit system gives you more for the next Wus ,correctly validated.
But , it's temporary , not permanent.When the situation comes back normal ,the credit system gives you the real credit expected for the computation accomplished.
I noticed the same behaviour for me , after endless Wus.
But the mechanism seems to be non linear(security on no-cheating maybe).

Erich
Send message
Joined: 18 Dec 15
Posts: 253
Credit: 1,942,248
RAC: 0
    
Message 5695 - Posted: 20 Nov 2016, 10:32:37 UTC - in response to Message 5688.

Perhaps , Erich , the credit system keeps in memory the fact several previous Wu failed in the past...

Thank you, Philippe, for your posting. Interesting approach.

How would you, though, comment the downwards development of points on the other PC, as described above in my posting ID # 5584?

PHILIPPE
Send message
Joined: 24 Jul 16
Posts: 84
Credit: 53,413
RAC: 0
    
Message 5697 - Posted: 20 Nov 2016, 16:23:32 UTC - in response to Message 5695.

Sorry , Erich ,there are not no more details on this period for this computer.The history of events has been deleted.
I can't answer you.
I try to read an other time, the page on "credit new", but i don't succeed to understand the system.
This system is perhaps not perfect , but the previous ones were not perfect too.
We are in the difficult state that we have to trust in a system ,we can't check and understand, at our level.
It's embarassing , not easy to live with , but do we have the choice?
And above all,everybody run with the same rules, so no one should be advantaged or disavantaged.
But you 're right ,sometimes ,looking at the credits assigned, it looks like "happy hours".Another person spoke in the forum about "a random generatory" with humour.
Too much factors, some are visible ,others not, take part in the assignement of credit.
Many persons keep silent about this because the perfect solution has not yet been found.
By the way , i'm not a forensic searcher ^_^

Erich
Send message
Joined: 18 Dec 15
Posts: 253
Credit: 1,942,248
RAC: 0
    
Message 5709 - Posted: 23 Nov 2016, 20:57:33 UTC

Date / task number / runtime / CPU time / points

Nov. 18: 7667999 / 39800 secs / 39290 secs / 613 points
Nov. 23: 7700616 / 39232 secs / 38619 secs / 403 points

the points are going down and down. No idea, why :-(

Nick Name
Send message
Joined: 22 Jun 14
Posts: 13
Credit: 135,078
RAC: 0
    
Message 5713 - Posted: 24 Nov 2016, 14:19:27 UTC - in response to Message 5709.

Date / task number / runtime / CPU time / points

Nov. 18: 7667999 / 39800 secs / 39290 secs / 613 points
Nov. 23: 7700616 / 39232 secs / 38619 secs / 403 points

the points are going down and down. No idea, why :-(

My experience with Credit New shows this is normal. Over time it grinds credit down to a fraction of what you get at the start.

This is not a project to do if you're interested in credit. When you take into account the enormous image files, VM overhead, and RAM and network load it might have the worst credit of them all.
____________
Team USA form

PHILIPPE
Send message
Joined: 24 Jul 16
Posts: 84
Credit: 53,413
RAC: 0
    
Message 5714 - Posted: 24 Nov 2016, 18:12:37 UTC - in response to Message 5713.

Nick Name wrote:
This is not a project to do if you're interested in credit. When you take into account the enormous image files, VM overhead, and RAM and network load it might have the worst credit of them all.

I don't think so.
If you compare credits assigned by projects,you earn more credits running atlas tasks than WCG tasks.
In my case , 25% ressources for atlas , 75% ressources for WCG , and i have nearly the same RAC for these 2 projects.
----------------------------------------------------------------------------------------------------------------------------------------------------------------
By the way , you can have more visibility about different earning credits at this page of boincstat site.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
Credit new is not perfect in real time but after some delays ,it appears to be not so unpredictable.(Adjustements on running times,between hosts more or less powerfull , and according to the non uniformity of jobs assigned to each of them, are made after a certain delay...,the checking proof is just missing...).

Nick Name
Send message
Joined: 22 Jun 14
Posts: 13
Credit: 135,078
RAC: 0
    
Message 5718 - Posted: 25 Nov 2016, 13:31:26 UTC - in response to Message 5714.

Nick Name wrote:
This is not a project to do if you're interested in credit. When you take into account the enormous image files, VM overhead, and RAM and network load it might have the worst credit of them all.

I don't think so.
If you compare credits assigned by projects,you earn more credits running atlas tasks than WCG tasks.
In my case , 25% ressources for atlas , 75% ressources for WCG , and i have nearly the same RAC for these 2 projects.
----------------------------------------------------------------------------------------------------------------------------------------------------------------
By the way , you can have more visibility about different earning credits at this page of boincstat site.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
Credit new is not perfect in real time but after some delays ,it appears to be not so unpredictable.(Adjustements on running times,between hosts more or less powerfull , and according to the non uniformity of jobs assigned to each of them, are made after a certain delay...,the checking proof is just missing...).

Odd, that stats page doesn't list ATLAS. Things may have changed since I last ran this project.

I still think the declining credit that was mentioned is to be expected.
____________
Team USA form

Profile Yeti
Avatar
Send message
Joined: 20 Jul 14
Posts: 699
Credit: 22,597,832
RAC: 0
    
Message 5719 - Posted: 25 Nov 2016, 13:38:44 UTC
Last modified: 25 Nov 2016, 13:39:21 UTC

The "drop" of credits is normal behaviour.

When a new Atlas-Application is started the credits given are unexpectable high.

As longer this appication runs the credit is going down until a normal range is reached.

I have seen this behaviour several times with each new Atlas-Application.

So, the Credits now are not unusual low, they have been unusual high in the past after handing out the latest application.

Don't ask me why, I can't tell or explain it.

1 · 2 · Next

Message boards : Number crunching : continuous drop of average credit since last week's WU problems