ATLAS@Home

ATLAS@Home is a research project that uses volunteer computing to run simulations of the ATLAS experiment at CERN. You can participate by downloading and running a free program on your computer.

ATLAS is a particle physics experiment taking place at the Large Hadron Collider at CERN, that searches for new particles and processes using head-on collisions of protons of extraordinary high energy. Petabytes of data were recorded, processed and analyzed during the first three years of data taking, leading to up to 300 publications covering all the aspects of the Standard Model of particle physics, including the discovery of the Higgs boson in 2012.

Large scale simulation campaigns are a key ingredient for physicists, who permanently compare their data with both "known" physics and "new" phenomena predicted by alternative models of the universe, particles and interactions. This simulation runs on the WLCG Computing Grid and at any one point there are around 150,000 tasks running. You can help us run even more simulation by using your computer's idle time to run these same tasks.

On this page you can see ATLAS monitoring of the tasks running on ATLAS@Home, as well as how ATLAS@Home ("BOINC") compares to the other sites on the WLCG.

The program you will download runs simulation software inside a virtual machine hosted by your computer. The virtual machine image is ~500MB but is only downloaded once. Each workunit downloads a small set of input data and runs for approx 1 to 2 hours depending on the computer's processor speed.

Please consult or post to the message boards to learn more about the work currently available or in case of problems.

The Physics of ATLAS@Home

No knowledge of particle physics is required, but for those interested in the physics processes simulated in ATLAS@Home as well as the ATLAS experiment itself visit the ATLAS web pages.

ATLAS@Home tasks progress, for each task (number of jobs done/total number of jobs):

  • Task mc15_13TeV CFilterBVeto_qsf4.simul (9160450): 17844/80000
  • Task mc15_13TeV CFilterBVeto_qsf025.simul (9160442): 91546/100000
  • Task mc15_13TeV CFilterBVeto_renorm4.simul (9160434): 99715/100000
  • Task mc15_13TeV CFilterBVeto_renorm025.simul (9160424): 79787/80000

Join ATLAS@Home

  • Read our rules and policies
  • This project uses BOINC. If you're already running BOINC, select Add Project. If not, download BOINC.
  • This project also requires VirtualBox to be installed.
    • Windows: VirtualBox is included in Windows distributions of Boinc and so does not have to be installed separately.
    • Mac and Linux: VirtualBox must be installed separately
    • VirtualBox 5.0 or 5.1 are the recommended versions. We have seen performance issues with older versions.
  • After choosing Add Project in Boinc Manager, choose ATLAS@home from the list of projects
  • Then either enter details for to set up a new account, or if you have previously registered enter the existing account details.
  • IMPORTANT:
    • A reasonably powerful modern 64-bit computer with at least 4GB of memory is required to run ATLAS@Home. Enabling 64-bit virtualisation may require some changes in BIOS settings.
    • In Boinc Manger set the % of processors used to be at most 50%, otherwise your computer may run out of memory.
    • The work units may use a lot of network bandwidth, so a slow internet connection may reduce work unit efficiency
    • The work units contact servers at CERN and outside CERN. If you are behind a firewall you will need to open access to these servers. Please contact us for details.
  • If you're running a command-line version of BOINC, create an account first.
  • If you have any problems, get help here.

User of the day

User profile Profile Vit Kliber         
I am cruncher from Czech National Team I manage http://stats.czechnationalteam.cz/.

News

Changes to the multicore app configuration
After analysis of results from the multicore app (see this thread) we have made the following changes

- Minimum VirtualBox version required is 5.0.0
- Maximum cores reduced from 12 to 8 (this can be overridden in app_config.xml but we have seen severe performance issues above 8 cores)
- Enforced that hardware acceleration is enabled on the host. This should stop the -148 ERR_EXEC errors because WU will not be sent to these hosts.

Please continue to provide your feedback on changes we make.
13 Sep 2016, 8:09:56 UTC · Comment


Tips on how to gain better performance on the ATLAS_MCORE app
According to our test, the CPU performance (We measure the CPU performance by seconds of CPU per event. For example, the current ATLAS job running on the ATlAS_MCORE app processes 100 events, if the overall CPU time for this job is 30000 seconds, then the CPU performance is 300 seconds/event) varies to the number of cores of the vm.

In the test, we also compared the CPU performance on different Virtualbox versions.

The test result can be seen here:



The above test is done on 2 hosts, HOST1 has HT(HypherThreading) enabled, while HOST2 has HT disabled. The test result is consistent despite of whether disabling/enabling HT of the CPU.

Also, we conclude a result from the ATLAS job statistics based on the jobs from over 1 month period. The following result shows the average CPU performance on different number of cores. (ATLAS_MCORE supports up to 12 cores for now).



The benefit of using more cores in one vm is to save memory usage, but using big number of cores can also significantly reduce the CPU performance. Our test result also concludes this is a case in all cloud computing platforms, not just on ATLAS@home.

In order to have a good tradeoff between memory usage and CPU performance, we advise that you configure the cores to each ATLAS_MCORE job(VM) according to the overall cores and memory allocated to BOINC.For example,if your host allocates 12 cores to BOINC, by default, ATLAS_MCORE creates a vm with 12 cores with 12.1GB memory, but if the host has enough memory, you can customize the usage with the app_config.xml file, i.e. each vm uses 6 cores with 7.3GB memory, so that your host runs 2 vms, and the overall memory usage is 14.6GB.

You can limit the MultiCore-App by using the app_config.xml (This file needs to be put in your project/atlasathome.cern.ch/ directory).

Below is an example to limit each ATLAS_MCORE job to use only 6 Cores:

<app_config> <app_version> <app_name>ATLAS_MCORE</app_name> <avg_ncpus>6.000000</avg_ncpus> <plan_class>vbox_64_mt_mcore</plan_class> <cmdline>--memory_size_mb 7300</cmdline> </app_version> </app_config>


You should change these two lines to your needs:

<avg_ncpus>4.000000</avg_ncpus> <cmdline>--memory_size_mb 7300</cmdline>


Memory usage calculated by the ATLAS_MCORE app is by this formula:

memory = 2500 + (800* NumerOfCores)

so it is 7300MB for 6 cores.

We will also make some changes on the server side very soon:
1. Require a minimum version(5.0.0) of Virtualbox for the ATLAS_MCORE app.
2. Limit the ATLAS_MCORE app to use at most 8 cores. 6 Sep 2016, 1:23:31 UTC · Comment

Performance issues with older Virtualbox
We recently noticed that with the ATLAS Multi Core application, there is a HUGE performance difference between the old and new version virtualbox, and this is especially obvious with big multi core (Core number >4 ).


As shown in this host with 12 Cores allocated to BOINC, the older version of virtualbox (4.2.16) uses almost 3 times of CPU time compared to the newer version(5.1.2) with the same kind of jobs.More results of other hosts from our database confirms this difference. And our database also shows around 7% of hosts still use older virtualbox(version<5.00). In order to avoid wasting the CPU, we advise that you upgrade the virtualbox to the most recent ones (5.1.2 or 5.1.4).

Also, in a few days, we shall setup a requirement on the server, so that hosts with older virtualbox(version<5.0.0) no longer receives the ATLAS Multi Core jobs.

Test results can be seen here

1 Sep 2016, 8:46:48 UTC · Comment


Moving multicore out of beta on Mon 29th August
We intend to move the multicore app out of beta on Monday the 29th August. This means that by default everyone will get both singlecore and multicore WU.

The main motivations for this are that multicore provides better CPU and memory efficiency, and we only have to manage a single stream of tasks instead of two.

If you do not want to run multicore you can disable it in your ATLAS@Home preferences in the option "Run only the selected applications"

By default the tasks will use as many cores as you have allocated to BOINC. For tuning the number of cores that the tasks will use you can create an app_config.xml file as described in this thread

At some point we plan to discontinue the singlecore app and run only multicore. The multicore app will also run on 1 core so you can still continue with ATLAS even with a single core.

Happy crunching!
The ATLAS@Home team
26 Aug 2016, 11:27:51 UTC · Comment


Multi-core version 1.04 with support for VirtualBox 5.1, OS X and graphics
Version 1.04 of the multicore app has been released with the following improvements:

- support for VirtualBox 5.1
- support for OS X (finally!)
- graphics (same as single core app)

Thank you for your continued support of ATLAS!
23 Aug 2016, 11:27:04 UTC · Comment


... more

News is available as an RSS feed   RSS