New multi-core application version 1.03


Advanced search

Message boards : News : New multi-core application version 1.03

1 · 2 · Next
Author Message
David Cameron
Project administrator
Project developer
Project tester
Project scientist
Send message
Joined: 13 May 14
Posts: 252
Credit: 2,028,082
RAC: 0
    
Message 4762 - Posted: 18 Jul 2016, 9:45:05 UTC

A new version of the multi-core app was released this morning which fixes the issue with the bootstrap script only setting up one core for each WU. It is recommended to abort older tasks using v1.01 if you want to take advantage of the fully working multi-core version.

Luigi R.
Send message
Joined: 6 Sep 14
Posts: 52
Credit: 123,536
RAC: 0
    
Message 4768 - Posted: 19 Jul 2016, 9:50:14 UTC

It looks something is wrong with my host. I'm trying multi-core app for the first time.

After 45 minutes my wu is at 20% and is using only 2 threads!

My app config:

<app_config>
<app_version>
<app_name>ATLAS_MCORE</app_name>
<max_concurrent>1</max_concurrent>
<avg_ncpus>5.000000</avg_ncpus>
<plan_class>vbox_64_mt_mcore</plan_class>
<cmdline>--memory_size_mb 6300</cmdline>
</app_version>
</app_config>

Profile tullio
Send message
Joined: 27 Jun 14
Posts: 256
Credit: 289,886
RAC: 0
    
Message 4770 - Posted: 19 Jul 2016, 15:13:37 UTC - in response to Message 4768.

How much CPU are you allowing in Computing preferences?
Tullio

Luigi R.
Send message
Joined: 6 Sep 14
Posts: 52
Credit: 123,536
RAC: 0
    
Message 4772 - Posted: 19 Jul 2016, 17:40:03 UTC - in response to Message 4770.
Last modified: 19 Jul 2016, 17:43:08 UTC

BOINC preferences -> use 100% of processors, use 100% cpu... id est 8 threads.

rbpeake
Send message
Joined: 27 Jun 14
Posts: 86
Credit: 8,794,961
RAC: 0
    
Message 4774 - Posted: 19 Jul 2016, 18:18:44 UTC - in response to Message 4772.
Last modified: 19 Jul 2016, 18:27:51 UTC

Did you reboot BOINC so the changes are read?
Also, why did you add the max concurrent line? That is not in the configuration explanation posted as the 2nd News item on the Main page.

Luigi R.
Send message
Joined: 6 Sep 14
Posts: 52
Credit: 123,536
RAC: 0
    
Message 4776 - Posted: 19 Jul 2016, 19:47:15 UTC
Last modified: 19 Jul 2016, 19:55:10 UTC

No, I didn't.

Firstly I read app_config, then I resumed work fetch for this project. It should be like a fresh start because I've not been computing Atlas@home for months.

But... you could be right. I downloaded ATLAS_MCORE while my config was 'use 38% of processors'.

You can see my client output (I removed useless lines).

19-Jul-2016 09:20:05 [ATLAS@home] Found app_config.xml
19-Jul-2016 09:20:05 [ATLAS@home] Your app_config.xml file refers to an unknown application 'ATLAS_MCORE'. Known applications: 'ATLAS'
9-Jul-2016 09:22:22 [ATLAS@home] work fetch resumed by user
19-Jul-2016 09:22:25 [ATLAS@home] Sending scheduler request: To fetch work.
19-Jul-2016 09:22:25 [ATLAS@home] Requesting new tasks for CPU
19-Jul-2016 09:22:27 [ATLAS@home] Scheduler request completed: got 1 new tasks
19-Jul-2016 09:22:29 [ATLAS@home] Started download of ATLASM_vboxwrapper_1.00_x86_64-pc-linux-gnu
19-Jul-2016 09:22:29 [ATLAS@home] Started download of ATLASM_vbox_job_1.00_mcore.xml
19-Jul-2016 09:22:30 [ATLAS@home] Finished download of ATLASM_vbox_job_1.00_mcore.xml
19-Jul-2016 09:22:30 [ATLAS@home] Started download of ATLASM_vbox_1.03_image.vdi
19-Jul-2016 09:22:54 [ATLAS@home] Finished download of ATLASM_vboxwrapper_1.00_x86_64-pc-linux-gnu
19-Jul-2016 09:22:54 [ATLAS@home] Started download of lmjLDmnXoion7jp7oou28CBqABFKDmABFKDmE3IKDmABFKDm7VVR6n_input.tar.gz
19-Jul-2016 09:23:06 [ATLAS@home] Finished download of lmjLDmnXoion7jp7oou28CBqABFKDmABFKDmE3IKDmABFKDm7VVR6n_input.tar.gz
19-Jul-2016 09:23:06 [ATLAS@home] Started download of lmjLDmnXoion7jp7oou28CBqABFKDmABFKDmE3IKDmABFKDm7VVR6n_EVNT.06479586._004157.pool.root.1
19-Jul-2016 09:24:00 [ATLAS@home] Started download of ATLASM_vbox_1.03_image.vdi
19-Jul-2016 09:24:00 [ATLAS@home] Started download of lmjLDmnXoion7jp7oou28CBqABFKDmABFKDmE3IKDmABFKDm7VVR6n_EVNT.06479586._004157.pool.root.1
19-Jul-2016 09:24:03 [ATLAS@home] Finished download of lmjLDmnXoion7jp7oou28CBqABFKDmABFKDmE3IKDmABFKDm7VVR6n_EVNT.06479586._004157.pool.root.1
19-Jul-2016 09:24:03 [ATLAS@home] Started download of rte_lmjLDmnXoion7jp7oou28CBqABFKDmABFKDmE3IKDmABFKDm7VVR6n.tar.gz
19-Jul-2016 09:24:04 [ATLAS@home] Finished download of rte_lmjLDmnXoion7jp7oou28CBqABFKDmABFKDmE3IKDmABFKDm7VVR6n.tar.gz
19-Jul-2016 09:24:04 [ATLAS@home] Started download of boinc_job_script.elmqWn
19-Jul-2016 09:24:05 [ATLAS@home] Finished download of boinc_job_script.elmqWn
19-Jul-2016 09:40:33 [ATLAS@home] Finished download of ATLASM_vbox_1.03_image.vdi
19-Jul-2016 11:02:22 [---] Reading preferences override file
19-Jul-2016 11:02:22 [---] Preferences:
19-Jul-2016 11:02:22 [---] max memory usage when active: 21879.13MB
19-Jul-2016 11:02:22 [---] max memory usage when idle: 23543.85MB
19-Jul-2016 11:02:22 [---] max disk usage: 34.21GB
19-Jul-2016 11:02:22 [---] Number of usable CPUs has changed from 3 to 8.
19-Jul-2016 11:02:22 [---] (to change preferences, visit a project web site or select Preferences in the Manager)
19-Jul-2016 11:02:22 [ATLAS@home] Starting task lmjLDmnXoion7jp7oou28CBqABFKDmABFKDmE3IKDmABFKDm7VVR6n_0



Max concurrent line was for personal future configurations (and maybe it doesn't work within app_version tags). Indeed after many hours while the ATLAS_MCORE app was using only 2 threads instead of 5, I modified the app_config to get max_concurrent tag to work.

<app_config>
<app>
<name>ATLAS</name>
<max_concurrent>2</max_concurrent>
<avg_ncpus>1.000000</avg_ncpus>
<plan_class>vbox_64</plan_class>
</app>
<app>
<name>ATLAS_MCORE</name>
<max_concurrent>1</max_concurrent>
<avg_ncpus>4.000000</avg_ncpus>
<plan_class>vbox_64_mt_mcore</plan_class>
<cmdline>--memory_size_mb 5300</cmdline>
</app>
</app_config>



However, for a mistaken app_config I would expect only 1 thread used, not 2. It's strange that BOINC and Virtualbox report 5 processors!





But top says:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
11270 luis 39 19 8637628 6,033g 5,965g S 193,5 26,0 1264:03 VBoxHeadle+

Cpu ~= 200% <==> 2 threads used



I won't reboot anything for now, I'm running BOINC on ramdisk. ATLAS app works fine.

Profile Yeti
Avatar
Send message
Joined: 20 Jul 14
Posts: 699
Credit: 22,597,832
RAC: 1
    
Message 4778 - Posted: 19 Jul 2016, 19:53:26 UTC - in response to Message 4776.

Firstly I read app_config, then I resumed work fetch for this project. It should be like a fresh start because I've not been computing Atlas@home for months.

Nope, it isn't. App_config's are only activated at Start of BOINC or if you choose "Read Config Files" in Menue of BOINC

Luigi R.
Send message
Joined: 6 Sep 14
Posts: 52
Credit: 123,536
RAC: 0
    
Message 4779 - Posted: 19 Jul 2016, 19:56:23 UTC - in response to Message 4778.
Last modified: 19 Jul 2016, 19:59:52 UTC

Nope, it isn't. App_config's are only activated at Start of BOINC or if you choose "Read Config Files" in Menue of BOINC


I manually read/activated configuration files. BOINC menu or boinccmd. See my client output.

zombie67 [MM]
Avatar
Send message
Joined: 18 Jun 14
Posts: 31
Credit: 1,175,117
RAC: 0
    
Message 4780 - Posted: 19 Jul 2016, 21:06:32 UTC - in response to Message 4778.

Firstly I read app_config, then I resumed work fetch for this project. It should be like a fresh start because I've not been computing Atlas@home for months.

Nope, it isn't. App_config's are only activated at Start of BOINC or if you choose "Read Config Files" in Menue of BOINC


BOINC does read app_config.xml when you select "read config files". You case see that happening in the event log.

I have noticed that with MT tasks, if you change the quantity of CPU threads to be used in app_config.xml, it will be implemented right away. But the BOINC Manager doesn't display the change until the next task starts.
____________
Dublin, California
Team: SETI.USA

Luigi R.
Send message
Joined: 6 Sep 14
Posts: 52
Credit: 123,536
RAC: 0
    
Message 4781 - Posted: 19 Jul 2016, 21:13:53 UTC
Last modified: 19 Jul 2016, 21:35:57 UTC

Maybe my task is never ending too.

After 12h9m is at 97.51% and checkpoint_cpu_time/cpu_time is 23h32m/23h48m.

No kernel panic is showed on vm screen (there is the usual localhost login).


UPDATE: finally ended!
http://atlasathome.cern.ch/result.php?resultid=6145306

David Cameron
Project administrator
Project developer
Project tester
Project scientist
Send message
Joined: 13 May 14
Posts: 252
Credit: 2,028,082
RAC: 0
    
Message 4782 - Posted: 20 Jul 2016, 7:43:33 UTC - in response to Message 4781.

This task was submitted before things were fixed in version 1.03, although you ran it with v1.03 so it should have worked. In the log there should be a line like

2016-07-20 05:40:04 (31563): Guest Log: ATHENA_PROC_NUMBER=4

This is the point where we tell the ATLAS software how many cores to use, but I don't see this line in your result log so it ran with the default value of 1. Do you see any improvement with newer tasks?

Luigi R.
Send message
Joined: 6 Sep 14
Posts: 52
Credit: 123,536
RAC: 0
    
Message 4784 - Posted: 20 Jul 2016, 9:04:38 UTC - in response to Message 4782.

Thanks for explanation.

I remember that my vm ran with 2 threads (not 1). It's weird.

Anyway I'm currently running ATLAS app (single-core). For what you said I will try ATLAS_MCORE again as soon as ATLAS queue is empty.

Profile Yeti
Avatar
Send message
Joined: 20 Jul 14
Posts: 699
Credit: 22,597,832
RAC: 1
    
Message 4786 - Posted: 20 Jul 2016, 10:37:29 UTC
Last modified: 20 Jul 2016, 10:37:48 UTC

Please check this WU: http://atlasathome.cern.ch/workunit.php?wuid=4820009

My Box had already cumulated something with 40 hours, but the cores seemed to be idling.

Looking into the VM gave me this screen:

captainjack
Send message
Joined: 28 Jun 14
Posts: 21
Credit: 182,001
RAC: 0
    
Message 4787 - Posted: 20 Jul 2016, 14:58:33 UTC

I'm getting "Waiting for memory" on both Windows 10 and Ubuntu 16.04.

On Windows 10, the box has 6 GB of RAM. I put in an app_config.xml to restrict it to 2 CPUs which should be small enough to run comfortably. Occasionally, it will start up and run for a while, but as soon as I move the mouse, it goes back to "Waiting for memory" and stops.

On Ubuntu, the box has 8GB of RAM, I put in an app_config.xml to restrict the Atlas task to 4 CPUs. The task started 15 minutes ago, it has an elapsed time of 2:34 and says it is "Waiting on memory". It has not yet created a virtual machine in VirtualBox.

Do I need to change something? Or is this typical?

Profile Yeti
Avatar
Send message
Joined: 20 Jul 14
Posts: 699
Credit: 22,597,832
RAC: 1
    
Message 4788 - Posted: 20 Jul 2016, 15:21:32 UTC - in response to Message 4787.

I'm getting "Waiting for memory" on both Windows 10 and Ubuntu 16.04.

On Windows 10, the box has 6 GB of RAM. I put in an app_config.xml to restrict it to 2 CPUs which should be small enough to run comfortably. Occasionally, it will start up and run for a while, but as soon as I move the mouse, it goes back to "Waiting for memory" and stops.

BOINC is exact doing, what you have set it up.

Check and change your preferences regarding memory usage while PC is in use and is not in use.

captainjack
Send message
Joined: 28 Jun 14
Posts: 21
Credit: 182,001
RAC: 0
    
Message 4790 - Posted: 20 Jul 2016, 15:52:06 UTC

Yeti,

Thanks for the help, I'm rolling along now. I hadn't changed the preferences for memory usage in so long that I didn't even think about them. I raised them up to 95% and both tasks are running now.

Thanks again,
captainjack

Luigi R.
Send message
Joined: 6 Sep 14
Posts: 52
Credit: 123,536
RAC: 0
    
Message 4792 - Posted: 20 Jul 2016, 22:17:12 UTC - in response to Message 4782.
Last modified: 20 Jul 2016, 22:19:56 UTC

David Cameron wrote:
Do you see any improvement with newer tasks?

Yes, it works.
http://atlasathome.cern.ch/result.php?resultid=6146092

Very large output though (~190mb).

Dave
Send message
Joined: 8 Jun 16
Posts: 2
Credit: 0
RAC: 0
Message 4793 - Posted: 21 Jul 2016, 4:15:44 UTC

Atlas@home does not work at all on my computer. Downloads files, takes a couple weeks to reach 100%, getting progressively slower as 100% is approached and then does nothing with the file. Am I supposed to tweak the system to make it work better? Think it will be easier to let my computer work on projects that work properly. Deleted and reinstalled the new version and still no difference. Moving on.

Profile Yeti
Avatar
Send message
Joined: 20 Jul 14
Posts: 699
Credit: 22,597,832
RAC: 1
    
Message 4794 - Posted: 21 Jul 2016, 4:43:41 UTC - in response to Message 4793.

Atlas@home does not work at all on my computer. Downloads files, takes a couple weeks to reach 100%, getting progressively slower as 100% is approached and then does nothing with the file. Am I supposed to tweak the system to make it work better? Think it will be easier to let my computer work on projects that work properly. Deleted and reinstalled the new version and still no difference. Moving on.

You should work through this Checklist: http://atlasathome.cern.ch/forum_thread.php?id=510

Luigi R.
Send message
Joined: 6 Sep 14
Posts: 52
Credit: 123,536
RAC: 0
    
Message 4818 - Posted: 24 Jul 2016, 18:33:42 UTC

Please add a new batch of workunits!

1 · 2 · Next

Message boards : News : New multi-core application version 1.03