Best Computer system for Atlas/LHC to build?


Advanced search

Message boards : Number crunching : Best Computer system for Atlas/LHC to build?

Author Message
Sparksy
Send message
Joined: 21 Jan 17
Posts: 6
Credit: 8,281
RAC: 0
  
Message 6081 - Posted: 25 Jan 2017, 23:36:23 UTC

Hello,
I'm wondering what the best computer would be to build for an Atlas@home or soon to be LHC@home server that I could build upon over time.

Would a bunch of second hand desktop PC's with heaps of ram be a comparable powerhouse?

I believe there is no GPU support so if that is correct I will just need CPU's linked together? Or will there be GPU support added at some time?

I have seen some cheap second hand servers like an IBM x3850 for $500 that I could collect over time to build a multi server but wonder if they are too slow? I'm having trouble finding speed comparisons between this and a desktop pc?

I also wonder would a Nvidia Tesla help me or is that classed as a GPU?

Obviously just a home project that I could build up over time or would it be better to save up buy something newer for a few thousand with more performance!!!

Thanks
Sparksy.

PHILIPPE
Send message
Joined: 24 Jul 16
Posts: 84
Credit: 53,413
RAC: 0
    
Message 6086 - Posted: 27 Jan 2017, 17:06:48 UTC - in response to Message 6081.
Last modified: 27 Jan 2017, 17:07:17 UTC

Hi Sarsky ,

i think your question is very interesting because nobody can answer correctly.
If you look at the top of hosts running for this project,you can notice they are rather different.

the 10 first have GenuineIntel Intel(R) Xeon(R) CPU E5-xxx.(16 to 56 processors)
powerful but perhaps expensive.
But if we try to define the best (higher rac,cheaper price,way of treating wu,os type,...)i think this one has a good ratio.
- 8 processors(Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz [Family 6 Model 60 Stepping 3])
- 16 Gigaoctets RAM (DDR3,DDR4,DDR5?)
- Linux
- no gpu (not useful for this project)
- 8-core multicore
I know my choice can be contested, i don't want to create a religion war (Intel-Amd,Windows-Linux,8-core-fewer core).
Everybody can choose and post another one if he explains why.

For the other part of your question , i am not enough updated to answer.

Profile Claus Varming Lund
Send message
Joined: 8 Mar 15
Posts: 23
Credit: 13,104,473
RAC: 0
    
Message 6087 - Posted: 27 Jan 2017, 20:58:11 UTC

Hi Sparksy
I agree with Philippe. I think you will get the bedst performance for money with a more recent i5 or i7 (4.gen or later.)
The Xeon CPU's runs with a lower temperature, and the models with many cores use a little less power per Mhz.
Xeon also supports ECC Memory (Error-correcting code memory), but I don't think that makes any deference in any of the LHC projects.

The Nvidia Tesla is a GPU, and is not supported in any LHC projects

I have an old server (Dell PowerEdge 1950) with two 4 core's cpu. (total 8 Core and no HT)
http://atlasathome.cern.ch/show_host_detail.php?hostid=41086
It is running Sixtrack at the moment.
I compared it with my Workstation some times ago. (Running with one 14 Core CPU - 28HT)
http://atlasathome.cern.ch/show_host_detail.php?hostid=48474

The old server only completed 25% of the work, compared to the more modern workstation, and the server used more then double the power. So thats why I recommend a more modern computer.

At last you should be aware that many rackserves are a bit noisy, (guess the fans can be chanced)
Just if you are living with some one, who do not like fan noise :-)

Tom*
Send message
Joined: 28 Jun 14
Posts: 118
Credit: 8,761,428
RAC: 0
    
Message 6088 - Posted: 27 Jan 2017, 21:34:19 UTC

http://atlasathome.cern.ch/top_hosts.php

My single proc i7-5820K CPU @ 3.30GHz running 3 ATLAS 3 core tasks

has the highest RAC of all systems with less than 16 CPU's

It does have 32Gig of memory but I also run one LHC@HOME Theory simultaneously

It was fairly in expensive at the time must be cheaper now.

No graphics on chip but IMHO that just sucks power and speed away from cpu

processing.

Tom aka Bill

Sparksy
Send message
Joined: 21 Jan 17
Posts: 6
Credit: 8,281
RAC: 0
  
Message 6089 - Posted: 27 Jan 2017, 22:38:39 UTC
Last modified: 27 Jan 2017, 22:39:07 UTC

Fantastic Info!
Thanks, I knew it was a little bit hard to answer and I've been reading up as much as I can.

Tom* your system seems to punching above it's weight there!

Claus Varming Lund you workstation seems to get through more data than some of the older dual CPU noisy server racks I was looking at! Does it have 130Gig of ram!

Philippe I thought I could bypass some single desktop computers with a old server but even that i7 you recommended does well in the passmark speeds and isn't noisy.
I was hoping to build a small scale supercomputer but it seems that I would just make a noisy wall that is inefficient and not that fast.
Claus's single CPU seems to be much faster than a IBM x3850 with dual xeon E5-2620, you would need 2 to go only slightly faster.

The IBM server was $1700 EA so for 2 = 3400 plus a rack to put them in, cabling etc.

or a simple desktop PC HP Z640 Tower Xeon E5-2697v3 for $4200. ex demo.


Tom* your cpu code is confusing me, I looked it up and it's a 6 core chip????
I can get a 6 core pc like that for $1400 and get in the top 10??
I must be getting something wrong.

Thanks again for all the info.
A friend gave me a i7-930 ill try to get running, haven't had any luck with ubuntu, it keeps saying the virtualbox isn't installed. which seems to be a popular problem. so i'll stick windows on it.

Cheers!

Sparksy

Tom*
Send message
Joined: 28 Jun 14
Posts: 118
Credit: 8,761,428
RAC: 0
    
Message 6090 - Posted: 27 Jan 2017, 23:29:17 UTC

Tom* your cpu code is confusing me, I looked it up and it's a 6 core chip????
I can get a 6 core pc like that for $1400 and get in the top 10??


Yup Its a six core Proc with HT its 12 CPU's The size of its local cache

seems to be a sweet spot currently for ATLAS I think I paid $800.00 total about

two or three years ago. The 3.3 GHZ cpu runs at 4.0 GHz in turbo mode

Built from the ground up usind TigerDirect and Amazon.

Well its not in the top 10 yet its 11:-)

Sparksy
Send message
Joined: 21 Jan 17
Posts: 6
Credit: 8,281
RAC: 0
  
Message 6091 - Posted: 28 Jan 2017, 1:01:07 UTC

Thanks, sounds like the winning price per calculations computer.
Might look for a few off those machines.

Profile Yeti
Avatar
Send message
Joined: 20 Jul 14
Posts: 699
Credit: 22,597,832
RAC: 0
    
Message 6093 - Posted: 28 Jan 2017, 16:47:28 UTC - in response to Message 6089.

Tom* your cpu code is confusing me, I looked it up and it's a 6 core chip????
Sparksy

Yes, it is a 6-Core-CPU plus Hyperthreading = 12 Cores to crunch, if you want

I have the same one in my box, but as it has taken part in a race at LHC@Home, the RAC at Atlas has fallen down. Normally it is something with 10.000 credits

My two most powerfull boxes that are fighting against each other:

http://atlasathome.cern.ch/show_host_detail.php?hostid=19234

versus

http://atlasathome.cern.ch/show_host_detail.php?hostid=36160

Don't know if you can see this: http://atlasathome.cern.ch/hosts_user.php?sort=expavg_credit&rev=0&show_all=0&userid=1735

Profile Yeti
Avatar
Send message
Joined: 20 Jul 14
Posts: 699
Credit: 22,597,832
RAC: 0
    
Message 6094 - Posted: 28 Jan 2017, 17:14:53 UTC

Take a look here: https://boincstats.com/en/stats/152/host/detail/36160/charts

Profile Claus Varming Lund
Send message
Joined: 8 Mar 15
Posts: 23
Credit: 13,104,473
RAC: 0
    
Message 6095 - Posted: 28 Jan 2017, 20:33:40 UTC - in response to Message 6089.

Claus Varming Lund you workstation seems to get through more data than some of the older dual CPU noisy server racks I was looking at! Does it have 130Gig of ram!


Yes :-) it has 128 GB Ram (8 x 16 DDR4-2133Mhz REG/ECC Quad Channel)

I'm using 52GB Ram for a RamDrive to the Boinc Data Folder.

Sparksy
Send message
Joined: 21 Jan 17
Posts: 6
Credit: 8,281
RAC: 0
  
Message 6096 - Posted: 28 Jan 2017, 23:49:06 UTC
Last modified: 29 Jan 2017, 0:07:27 UTC

Some great computers!

Ram does seem to help by letting you run as many simulations at the same time as possible. From what I have seen often ram runs out before cpu.

Yeti, your Xeon processor seems slower than the i7 but you have a little more ram showing in the Xeon.
Very interesting they are matched so closely!

Looks like the noisy server is not a great idea.
Building a current PC sounds like what ill do.

I am learning quite a lot and want to thank you all for the comments.
Having only started with doing anything like this a week ago I've still got heaps to learn.

Borrowed another computer to get up and crunching which is taking a while, swapped psu, replaced hdd with ssd and faulty video card, finally got it running to find "Error while computing" for every task or "Computational Error" after 32min of 8 cpu task, and blank white screen on VM Console ...still working on this one.

PHILIPPE
Send message
Joined: 24 Jul 16
Posts: 84
Credit: 53,413
RAC: 0
    
Message 6097 - Posted: 29 Jan 2017, 9:04:59 UTC - in response to Message 6096.

Hi , Sparsky,

In the logs of your tasks is written :

NOTE: VM failed to enter an online state within the timeout period.
This might be a temporary problem and so this job will be rescheduled for another time.

At the end of your log appears:
upload failure: <file_xfer_error>
<file_name>My3NDmiiXrpnDDn7oo6G73TpABFKDmABFKDmsEGKDmABFKDmJLscCo_0_r1385285500_ATLAS_result</file_name>
<error_code>-161 (not found)</error_code>
</file_xfer_error>

It appears when boinc can't be linked with virtualbox.
- Check if you have removed HyperV in your windows 10 professional.
- Install latest virtualbox version and the pack extension(same version)
- Open virtualBox and delete all the vm failed (marked with a red sign) if they are present.
-Reboot your computer to initialize.
- you need a pemanent connection to internet

You have to follow the notes in your logs , they can help you to understand what is right and what is wrong.

Hope it helps you...

PHILIPPE
Send message
Joined: 24 Jul 16
Posts: 84
Credit: 53,413
RAC: 0
    
Message 6098 - Posted: 29 Jan 2017, 13:28:14 UTC - in response to Message 6097.

just to give you more information:

this thread relating to the same error

chapter on the preferences :
local preference and the web preferences

It 's not always easy to understand the causes of troubles but it's interesting to try to solve them.

Sparksy
Send message
Joined: 21 Jan 17
Posts: 6
Credit: 8,281
RAC: 0
  
Message 6103 - Posted: 30 Jan 2017, 19:26:44 UTC

Thanks Phillipe,
Worked through a bunch of your fixes.
I had worked through Yeti's ones at the top of this topic too.
The computer also had a major win 10 update and I removed some old win 7 lan drivers.
Seems to be up and running though I'm not sure which one fixed it.
Thanks a lot!

The old I7 is running away and looks like ill need a upgraded "quieter" cpu cooler! the stock one is running quite hard.

Sparksy
Send message
Joined: 21 Jan 17
Posts: 6
Credit: 8,281
RAC: 0
  
Message 6121 - Posted: 6 Feb 2017, 23:14:03 UTC

Spew, Looks like I'm unable to get past this 161 error.
Not sure I understand how boinc and vbox aren't talking to eachother properly.
Tried as many as possible, because I had it working at one stage I formatted my computer and started from scratch but still nothing. Possibly my knowledge of IT and opening router and firewall ports are letting me down as I don't know how to make sure they are open. I'll have to spend some more time learning, (watching you tube) about these settings.
Hyper V is unchecked in windows.
Reset computers to web preferences, I had them set to different computer preferences.

I have been successful with LHC@home cms calculations but I'm well aware Atlas requires a lot more settings to be correct than CMS.

I'll have a break and run cms for a few days and get back into configuring for Atlas when I have some more time.

Profile [AF>EDLS]GuL
Send message
Joined: 3 Jan 15
Posts: 4
Credit: 503,824
RAC: 0
    
Message 6134 - Posted: 9 Feb 2017, 18:53:15 UTC

Hi all,
Thanks for this interesting discussion.

I was wondering first if VT-D extensions where more efficient than VT-X ones, second what is the influence of RAM speed and L2 cache size.

Indeed, I noticed on this host which is a i5 4670K @ 4.4 GHz with 16 GB of RAM @ 1800 MHz that running more than 2 single-core WU or more cores than 2 on multi-cores WU will jerk the computer, like when using HyperPI. Is VirtualBox memory bandpass limited or L2 cache bandpass limited ?

Thank you for your lightening advices.
____________

PHILIPPE
Send message
Joined: 24 Jul 16
Posts: 84
Credit: 53,413
RAC: 0
    
Message 6139 - Posted: 10 Feb 2017, 18:09:02 UTC - in response to Message 6134.

Hi , [AF>EDLS]GuL ,

after reading the feature of vt-d ,in wikipedia ,if i have well understood,it seems that vt-d (a variant of vt-x) which allows a direct access to different chipsets (graphic cards,ethernet,audio,...) via the data bus is effectively faster and more reliable than vt-x but it is not possible on all computers.Motherboard and chipsets have to be compatible.
I find here a list of cpu compatible.
Because the VM is executed inside the ram memory ,(only the save points and the logs are recorded on the hard disk during the running),the speed of the ram and the bus linked to the cpu are an important factor to increase flops.
Apparently , the L2 cache size can't be evaluated properly to find an optimum because it depends the way the cpu expects to predict future actions and for instance the size of small loops present in the project algorythm .

You can see cpu gigaflops here and there to compare eventually various types of configuration.

For the other part of your question, the behaviour with multi-cores ,i can't answer you ,i can only run one single core in my computer because lack of ram memory (4GBytes only).

Regards.

Profile [AF>EDLS]GuL
Send message
Joined: 3 Jan 15
Posts: 4
Credit: 503,824
RAC: 0
    
Message 6140 - Posted: 10 Feb 2017, 19:43:25 UTC - in response to Message 6139.


Because the VM is executed inside the ram memory ,(only the save points and the logs are recorded on the hard disk during the running),the speed of the ram and the bus linked to the cpu are an important factor to increase flops.


Thank you for your interesting answer.
Does anybody has compared different ram speeds on Atlas ?
Cheers
____________

Message boards : Number crunching : Best Computer system for Atlas/LHC to build?