Single core tasks finished


Advanced search

Message boards : News : Single core tasks finished

Author Message
David Cameron
Project administrator
Project developer
Project tester
Project scientist
Send message
Joined: 13 May 14
Posts: 248
Credit: 2,026,552
RAC: 476
      
Message 5591 - Posted: 10 Nov 2016, 10:50:14 UTC
Last modified: 11 Nov 2016, 8:15:06 UTC

You may have noticed that there are no single core tasks left in the queue. This is deliberate as part of our aim to simplify things by having a single app which is multicore only.

We know that there are people who like to run only single-core WU but it is possible to run multicore WU in a single-core mode through the use of app_config.xml. (set

<avg_ncpus>1</avg_ncpus>
)

This is not a perfect solution so if there are serious objections to deprecating the single core app we can do some tricks to send multicore WU to the single core app which will then run in single-core mode. Please let us know what you think.

captainjack
Send message
Joined: 28 Jun 14
Posts: 21
Credit: 182,001
RAC: 0
    
Message 5592 - Posted: 10 Nov 2016, 16:01:13 UTC

David,

When I run multicore tasks limited to 1 thread, BOINC thinks the tasks take up almost all available memory and not much else runs.

For example: my Linux machine shows to have 7.7 GB of available memory. When I download an ATLAS multicore task, the list on the web page shows that the task has a core number of 6. An app_config is in place to limit the task to one cpu and 3300 mb of memory. Computing preferences are set to allow BOINC to use 100% of available memory. When the system is running 2 GPUGRID tasks, there are 1.7 GB of memory in use (GPUGRID plus O/S). When I start up an ATLAS task using one thread, it uses 3.4 GB of memory. The system now shows that 5.2 GB of memory in use. The WCG tasks in the queue use 86 MB of memory. Only one is allowed to run and BOINC thinks it is out of memory. So an 8 thread machine will only run 2 CPU tasks and the System Monitor shows that it has 2.5 GB of memory available.

Are there any plans to adjust the memory requirement used by BOINC for scheduling purposes?

Profile Yeti
Avatar
Send message
Joined: 20 Jul 14
Posts: 696
Credit: 22,597,832
RAC: 6,119
      
Message 5593 - Posted: 10 Nov 2016, 16:37:51 UTC

As I already mentioned in this thread: http://atlasathome.cern.ch/forum_thread.php?id=608
MultiCore-WU doesn't run on this box; the error-message shown is posted in the thread.

Please, provide a solution for this problem or light me how to solve this

Thank you

Profile Sir Thomas W. Kilburn
Send message
Joined: 24 May 15
Posts: 4
Credit: 87,962
RAC: 39
      
Message 5596 - Posted: 10 Nov 2016, 22:35:06 UTC - in response to Message 5591.

I like it

Profile tullio
Send message
Joined: 27 Jun 14
Posts: 256
Credit: 288,515
RAC: 97
      
Message 5597 - Posted: 11 Nov 2016, 1:34:49 UTC - in response to Message 5591.

The app_config.xml seems to work on my 2 core Opteron 1210 running SuSE Leap 42.1 Linux, at least one SETI@home GPU task is running alongside Atlas@home.
Tullio

nocelc
Send message
Joined: 21 Mar 16
Posts: 3
Credit: 163,307
RAC: 71
      
Message 5602 - Posted: 11 Nov 2016, 12:04:54 UTC

The multi-core version always fails on my machine while the single core works. Even with the app-config.xml set to 1 core.

PHILIPPE
Send message
Joined: 24 Jul 16
Posts: 80
Credit: 53,413
RAC: 172
      
Message 5605 - Posted: 11 Nov 2016, 15:39:24 UTC - in response to Message 5602.

Can you give us more details ?
What appears in your events diary in boinc manager?
It should display this pieces of information :


11/11/2016 11:38:56 | | OS: Microsoft Windows 10: Core x64 Edition, (10.00.14393.00)
11/11/2016 11:38:56 | | Memory: 3.67 GB physical, 6.29 GB virtual
11/11/2016 11:38:56 | | Disk: 869.91 GB total, 804.35 GB free
11/11/2016 11:38:56 | | Local time is UTC +1 hours
11/11/2016 11:38:56 | | VirtualBox version: 5.1.6
11/11/2016 11:38:56 | ATLAS@home | Found app_config.xml
11/11/2016 11:38:56 | | Config: fetch on update

What has you written in your app_config.xml file ?
It should be,if you want run a 8 cores work unit at a time :

<app_config>
<app_version>
<app_name>ATLAS_MCORE</app_name>
<avg_ncpus>8.000000</avg_ncpus>
<plan_class>vbox_64_mt_mcore</plan_class>
<cmdline>--memory_size_mb 9000</cmdline>
</app_version>
</app_config>

But if you want to go on multi-core work units with an alone core, you can put into, as you have 32GB of ram memory :

<app_config>
<app_version>
<app_name>ATLAS_MCORE</app_name>
<avg_ncpus>1.000000</avg_ncpus>
<plan_class>vbox_64_mt_mcore</plan_class>
<cmdline>--memory_size_mb 3300</cmdline>
</app_version>
</app_config>

8 multicores (1 cpu) tasks will be executed.
It's up to you to choose what you prefer.(1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 multi-core tasks) You have enough ram to do anything you want...
Not necessary for you to have an app_config.xml file except if you want to customize with particular needs.
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
But first of all, you have to allow in the preferences of your account :
ATLAS Simulation Running on Multiple Core
Because I don't see any multi-core wus in your tasks page, even aborted or with errors.

BamaMath
Send message
Joined: 10 Aug 16
Posts: 12
Credit: 1,318,794
RAC: 2,074
      
Message 5620 - Posted: 12 Nov 2016, 16:59:17 UTC

Why do single-core jobs continue to download if there are none in the queue? I continue to have 8 8-core jobs and 8 1-core jobs in my local queue.

Jesse Viviano
Send message
Joined: 20 Dec 15
Posts: 16
Credit: 328,453
RAC: 760
      
Message 5622 - Posted: 12 Nov 2016, 17:49:48 UTC - in response to Message 5620.

We forum users can't really help you if you keep your computers hidden. However, there are plenty of old work units still left, and their tasks sometimes fail or time out. New tasks are created to replace them and still need to be processed.

BamaMath
Send message
Joined: 10 Aug 16
Posts: 12
Credit: 1,318,794
RAC: 2,074
      
Message 5628 - Posted: 13 Nov 2016, 21:51:58 UTC - in response to Message 5620.

So given no answer, I've suspended the 8-core jobs until the 1-core jobs finish. After that I'll abort any more 1-core jobs received.

David Cameron
Project administrator
Project developer
Project tester
Project scientist
Send message
Joined: 13 May 14
Posts: 248
Credit: 2,026,552
RAC: 476
      
Message 5635 - Posted: 14 Nov 2016, 10:17:09 UTC - in response to Message 5628.

There are around 3000 WU still running in the single-core app but those are probably either stuck on a PC that is not running or bad WU which fail and get retried many times. So you still have a good chance to get single-core WU but I would recommend in your case running only multi-core (rather than aborting you can configure to only run multi-core in your account preferences).

BamaMath
Send message
Joined: 10 Aug 16
Posts: 12
Credit: 1,318,794
RAC: 2,074
      
Message 5640 - Posted: 14 Nov 2016, 23:04:05 UTC - in response to Message 5635.

Thank you. I set up app_config.xml to run only 8-core as follows:

<app_config>
<app>
<name>ATLAS</name>
<max_concurrent>1</max_concurrent>
</app>
<app_version>
<app_name>ATLAS_MCORE</app_name>
<avg_ncpus>8.000000</avg_ncpus>
<plan_class>vbox_64_mt_mcore</plan_class>
</app_version>
<project_max_concurrent>1</project_max_concurrent>
</app_config>

Correct? Note that I deleted the line <cmdline>--memory_size_mb 3300</cmdline>. I've got 32G available and Atlas is welcome to whatever is needed. If I need that back, what's the rule of thumb?

Message boards : News : Single core tasks finished