Folding on a Ramdisk

musky

[H]ard|DCer of the Year 2012
Joined
Dec 14, 2009
Messages
3,154
THIS ONLY WORKS WITH THE 6.34 CLIENT INSTALLED PER THE INSTRUCTIONS IN THE PREVIOUS UBUNTU SETUP GUIDE (Ubuntu 10.10.) IF YOU INSTALLED UBUNTU FOLLOWING THE DIRECTIONS IN THE CURRENT GUIDE (Ubuntu 12.04 with the fahinstall.sh scipt), YOU SHOULD NOT FOLLOW THIS GUIDE - IT IS ALREADY INSTALLED.

ALSO, IT DOES NOT WORK WITH v7, NOR DO I PLAN ON MAKING IT WORK WITH v7


These are instructions for setting up a temporary file system to use for running F@H - basically a ramdisk. The reason to to this is that it makes disk write times almost instantaneous. The down side is that it is a temporary file system, meaning it is lost when the system powers down or reboots. Any risk and inconvenience of folding on a temporary file system is remedied by using scripts and a cron job to keep everything backed up and to automatically set everything up correctly when powering up your system. All of this has been validated pretty thoroughly, and it is definately worth the time to set up on any Linux machine.

***NOTE 1*** This was tested on Ubuntu 10.10 and 12.04 LTS. It should work on any Ubuntu version. The only issue you might run into is with rc.local and/or the /etc/init.d script not executing in later versions of Desktop (12.04 and 12.10.) If you run into issues with the startup script running, let me know.
***NOTE 2*** Based on my testing, the easiest thing to screw up is to not replace "<user>" with your Linux user in the steps where this is required (rc.local edit, fstab edit, /etc/init.d script, and smb.conf file.) Please be aware of this and don't forget to edit the lines after you copy/paste them.

First, remove any backup/restore scripts from /usr/bin if they exist. If you get a "file not found" type of error, that is fine.
Code:
sudo rm -f /usr/bin/fahbackup
sudo rm -f /usr/bin/fahrestore
Download these scripts to /usr/bin so they can be used from anywhere. You will need to use sudo to download and chmod them.
Code:
cd /usr/bin
sudo wget http://darkswarm.org/fahtools/fahbackup
sudo wget http://darkswarm.org/fahtools/fahrestore
sudo chmod +x fahbackup
sudo chmod +x fahrestore
Create a backup by running the fahbackup script.
Code:
fahbackup
Verify the backup - you will now have a fahbackup-data directory in your home directory. In that directory, you will have a directory called current. In that directory, you should have a file named <cureent date/time>.tar. Entering this:
Code:
ls ~/fahbackup-data/current
should return something like this, only with the current date/time as the file name:
Code:
2012-11-29-0734.tar
If it exists, you are ready to proceed. First, stop the client.
Code:
pkill fah6
Next, delete the contents of your fah directory (don't worry - you just backed everything up.)
Code:
rm -r ~/fah/*
Get your user ID and group ID - run the following two commands and note the results (if you have a typical Ubuntu install with a single user, these will both be 1000):
Code:
id -u
id -g
Update /etc/fstab to mount a tmpfs filesystem on boot.
Code:
sudo nano /etc/fstab
Add this line to the bottom of the file, replacing <user> with your Linux user, <userd ID> with your user ID, and <group ID> with your group ID:
Code:
tmpfs   /home/<user>/fah  tmpfs   rw,uid=<user ID>,gid=<group ID>      0       0
Save the file (Ctrl X - Y - Enter) and mount the new tmpfs file system to the fah directory
Code:
sudo mount -a
Restore you fah directory with the fahrestore script
Code:
fahrestore
Check the contents of your ~/fah directory to make sure everything looks correct. Entering:
Code:
ls ~/fah
should return something like this:
Code:
client.cfg      FAHlog.txt            queue.dat                 thekraken-prev.log
fah6            langouste-helper.sh   thekraken-FahCore_a3.exe  unitinfo.txt
FahCore_a3.exe  machinedependent.dat  thekraken-FahCore_a5.exe  work
FahCore_a5.exe  MyFolding.html        thekraken.log
If it returns nothing, the restore did not work

Update /etc/rc.local so it restores the directory when the system boots up.
Code:
sudo nano /etc/rc.local
Add this line before the exit 0 line. Again, replace <user> with your Linux user.
Code:
sudo -u <user> fahrestore
Save your changes (Ctrl X - Y - Enter)

Set up a cron job to run the backup script hourly. The backup process does not affect folding performance, so you should not see any ppd drop running it hourly. Also note that you need to run this as your Linux user - DO NOT run this with sudo.
Code:
crontab -e
If you haven't used crontab -e before, you will be prompted for an editor - choose nano. Add this line to the bottom of the file that opens. If the file is empty, that is fine - add this as the first line.
Code:
00 * * * * fahbackup > /dev/null 2>&1
Save your changes (Ctrl X - Y - Enter)

Next, you want to set the system up to create a backup when you shut down, reboot, or if the cron daemon stops. To do this, first make a simple script in /etc/init.d
Code:
sudo nano /etc/init.d/fahbackup-rc
The script itself is just two lines. Again, replace <user> with your Linux user.
Code:
#!/bin/bash
sudo -u <user> fahbackup
Save the file (Ctrl X - Y - Enter), then make the script executable
Code:
sudo chmod +x /etc/init.d/fahbackup-rc
Finally, add symlinks to this script in rc0.d, rc1.d, and rc6.d:
Code:
sudo ln -s  /etc/init.d/fahbackup-rc /etc/rc0.d/K10fahbackup-rc
sudo ln -s  /etc/init.d/fahbackup-rc /etc/rc1.d/K10fahbackup-rc
sudo ln -s  /etc/init.d/fahbackup-rc /etc/rc6.d/K10fahbackup-rc
One note here - shutdowns and reboots can take up to a couple of minutes with these backup scripts in place. Just be aware that if you see your machine seem to hang on reboot, it is probably normal. Just give it a couple of minutes and everything should be fine.

Samba setup: If you are using a Desktop Linux version and you used the GUI tool to share your folding directory for remote monitoring with HFM, you will need to switch to using the smb.conf file to configure file sharing. The GUI tool does not work with tmpfs file systems, so anything that relies on this share (such as HFM) will not work. Don't worry though - the configuration is not that difficult.

First, back up your current samba config file (smb.conf)
Code:
sudo mv /etc/samba/smb.conf /etc/samba/smb.conf.bak
Next, create a user share - replace <user> with your user:
Code:
net usershare add fah /home/<user>/fah "" "Everyone:f" "guest_ok=y"
Open /etc/samba/smb.conf in nano:
Code:
sudo nano /etc/samba/smb.conf
Find this line (it is towards the top):
Code:
#      security = user
Delete the # and change it to:
Code:
       security = share
Save the file (Ctrl X - Y - Enter) and reload samba
Code:
sudo service smbd reload
Make sure your share is accessible from your HFM machine. The path for HFM should be something like \\machinename\fah.

You should verify that your backup and restores scripts work on a reboot. Check the time on your last backup, and verify that you have a restored fah directory. Reboot your system, then verify that you still have a restored fah directory and that your latest backup has the date and time you rebooted. If you do, you are set. Restart your fah client like you normally do. You should also make sure your crob job is working by making sure that your latest backup (in ~/fahbackup-data/current/) updates each hour. If it does, you are finished!

While this looks compilcated, it really isn't. I would expect it to take 15 minutes or less per machine, even with reboot testing. It is definitely worth the time to set up. If you run into any problems, the best way to get help will be in irc (irc.dp.cx - #hardocp) - ask for tear or musky. Otherwise, posts any issues here. Any feedback on these instructions is also appreciated.
 
Last edited:
It is definitely an improvement folding on a ramdisk and well worth the effort. Thanks for the good write up musky :)
 
pretty new to folding here, what are the advantages to ramdisk folding? Note i am not folding bigadv
 
Looks like another excellent musky guide. Out of curiosity, what kind of real-world benefit are we looking at here in ppd terms?
 
It cuts down a lot on the time for post processing the work unit and getting it ready to send back.
 
Warning - I seldom follow directions correctly and I know DOS. A dangerous combo in Linux.

Anyhow, if you install U64-10.04LTS, then go to the folding.stanford.edu site today and install the FAHClient and FAHControl,

Now you open Applications>Accessories>Terminal from the top left pulldown menus (GUI).

You are now at:

tyan@tyan-desktop:~$ (tyan is USER, tyan-desktop is the machine name)

then do both rm commands from the instructions.

then cd /usr/bin

then the wgets

then the chmods

everything is cool, and everything works.

Now I check to see they are there.

tyan@tyan-desktop:/usr/bin$ ls fah*
fahbackup fahrestore

Now I give it the backup command.

tyan@tyan-desktop:/usr/bin$ fahbackup
/home/tyan/fah empty. Cannot run.

Now I check if this is the FAH install directory.

tyan@tyan-desktop:/usr/bin$ ls FAH*
FAHClient FAHControl FAHCoreWrapper
tyan@tyan-desktop:/usr/bin$

So the FAH directories are here, so are the scripts, so what is wrong?
 
Don't use v7. This won't work with v7, nor will it ever. I'll update the first post.

If you want an easy way to install v6.34, see the Ubuntu install guide starting with downloading the fahinstall.sh script. That should work on 10.04 - let me know if it does not.
 
Yup, I just posted that so that others who do the "suggested" default install will know they didn't screw up.

No slight intended.
 
Last edited:
Out of curiousity, is there a "DOS PATH" command in Linux?

That might be all that is needed to adjust for current and future changes.

For the non-DOS guys, it allows you to be in the wrong directory when executing programs.
 
Out of curiousity, is there a "DOS PATH" command in Linux?

That might be all that is needed to adjust for current and future changes.

For the non-DOS guys, it allows you to be in the wrong directory when executing programs.


Code:
pwd

Some people like the path to be part of the command prompt:

Code:
export PS1="`pwd` > "
 
Yup, I just posted that so that others who do the "suggested" default install will know they didn't screw up.

No slight intended.

It is all about whether you are talking about Stanford's default install or mine. I assumed that if you were looking at my ramdisk guide, your starting point was my install guide. That is a poor assumption on my part. Sorry for the confusion.
 
What sort of PPD gain can we get from using this Ramdisk method?

It is a worth while bump. The bigger/larger the system, the more PPD increase. Remember, bonus is based on start of download to end of upload.
 
Musky (or anyone that knows),

In the RamDisk procedure above, when it says:

Finally, add symlinks to this script in rc0.d, rc1.d, and rc6.d:

Code:
sudo ln -s /etc/init.d/fahbackup-rc /etc/rc0.d/K10fahbackup-rc
sudo ln -s /etc/init.d/fahbackup-rc /etc/rc1.d/K10fahbackup-rc
sudo ln -s /etc/init.d/fahbackup-rc /etc/rc6.d/K10fahbackup-rc

Was I supposed reopen a script file via "sudo nano /etc/init.d/fahbackup-rc" and add those (3) lines that are shown above? Forgive my Linux ignorance. I didn't understand if I was just supposed to execute those three lines of code at a command prompt or actually copy/paste them into a script file.

If I try to just execute them at a command prompt, I get a response that says
"failed to create symbolic link '/etc/rc0.d/K10fahbackup-rc': File exists"
I get that same type of response trying to execute the other two lines of code too.

Thanks for any help.

PS: How do we check/know that the RAMDisk is actually running as intended? Is there somewhere I can check that in my devices? I never did clearly understand which of the steps I followed in the RAMDisk Guide actually created the RAMDisk. I just want to make sure that when I start a Folding Job that it is actually running from RAM and not my HDD. Thanks.
 
Last edited:
If you used current Ubuntu setup script (installed last week of March 2013 or later) then
ramdisk is already there and no action needs to be taken.
I suppose that first post could be edited to reflect that better [it doesn't really cut it
as far as I'm concerned].

The lines you're asking about (symlink creation) are to be executed in the command prompt
(just like you did).
File exists means that a file (most likely symlink) of such name already exists == suggests
that you tried to run them more than once or that install script had already created them for
you.

I would think that changing ln -s to ln -fs could be a healthier choice (for new users).

Either way, you can run fahdiag to check if your setup is correct.

fahdiag setup:
Code:
cd $HOME
rm -f fahdiag.sh
wget http://darkswarm.org/horde/fahdiag/fahdiag.sh
chmod +x fahdiag.sh

running fahdiag:
Code:
./fahdiag.sh | pastebinit

Then share resulting URL with us.
 
Thank-you Tear for the help on my questions. :cool:

I did follow the new/latest Ubuntu Install Guide (step by step) to install 12.04 LTS and I should have mentioned that as a baseline. In that Ubuntu Guide, I found in several areas where I was being asked to edit a file and type-in some params, that they were already there exactly as requested in the Guide. So the Ubuntu setup script must have already done those things for me. The new script is quite well done I must say!

I did Fold a standard SMP (8531 WU) just as a test on my 4P and found that I was getting a 10sec improvement in my TPF vs my prior installation if Linux. I'll give a Bigadv a go next to see what it does for comparison (if I can get one...spotty availability lately).

I ran what you asked (the fahdiag) and here is the resulting URL. I don't have Langouste installed, so it reported "file not found" when I ran the diag.
http://paste.ubuntu.com/5684466/

Again, really do appreciate the help today and if I can be of any more help, just let me know.
 
texinga, you have one things to clean up - in /etc/rc.local, you have these two duplicate lines:

sudo -u rick fahrestore #[H]ardOCP
sudo -u rick fahrestore

Remove the one that doesn't end in #[H]ardOCP (sudo nano /etc/rc.local).

Just out of curiosity, why did you think that you needed to follow this guide after your 12.04 install? From that guide"

"This script will set up everything you need to run Folding@Home on a ramdisk, including thekraken wrapper, backup/restore scripts, and samba file sharing for remote access."

I just want to know what I need to do to make this clearer.

Thanks
 
Musky, thank-you for reviewing my content. I do recall entering that duplicate entry into /etc/rc.local and I know what happened there. I did not catch/see the similar entry at the top with HARDOCP at the end of it. It too must have already been put there and I didn't notice it.

As to why I followed the "Folding on a Ramdisk" Guide, I did not realize that you had all that stuff already done in the new script. I thought that I had to follow the Ubuntu 12.04 LTS Guide and then also follow the Ramdisk Guide.

I really love what you did with that script with already having the Cores, The Kraken, etc done for us. I'd say that this is the easiest way to get all that accomplished that I've seen...and I've used your previous Guides too. Just trying to say that what you have done for us is very much appreciated! :cool:
 
I had some trouble with setting up a ramdisk, here:

Code:
vidar@LinuxRigg12:/usr/bin$ sudo nano /etc/fstab

vidar@LinuxRigg12:/usr/bin$ sudo mount -a

mount: wrong fs type, bad option, bad superblock on tmpfs,
       missing codepage or helper program, or other error
       (for several filesystems (e.g. nfs, cifs) you might
       need a /sbin/mount.<type> helper program)
       In some cases useful info is found in syslog - try
       dmesg | tail  or so
I guess <user ID>, gid = <Group ID> is user ID from the log, and that Group ID is the teamnumber?
 
I had some trouble with setting up a ramdisk, here:

Code:
vidar@LinuxRigg12:/usr/bin$ sudo nano /etc/fstab

vidar@LinuxRigg12:/usr/bin$ sudo mount -a

mount: wrong fs type, bad option, bad superblock on tmpfs,
       missing codepage or helper program, or other error
       (for several filesystems (e.g. nfs, cifs) you might
       need a /sbin/mount.<type> helper program)
       In some cases useful info is found in syslog - try
       dmesg | tail  or so
I guess <user ID>, gid = <Group ID> is user ID from the log, and that Group ID is the teamnumber?

From the guide:

Get your user ID and group ID - run the following two commands and note the results (if you have a typical Ubuntu install with a single user, these will both be 1000):
Code:
id -u
id -g

id -u will return the user ID, and id -g will return the group ID. These are the two values you use in the fstab entry. If you still have issues, please show me the output of
Code:
cat /etc/fstab
 
Hi and thank you for answering!

I did find your fahinstall script here: http://hardforum.com/showthread.php?p=1037125470 and used that instead. If I understand it right, it do installation of fah, kraken and a ramdisk as well.

The installation went without error and everything looks fine. But when I start fah it is folding a few % and then it freezes the machine. There is no error message or anything on the screen, it simply freezes and I have to restart the machine the hard way.

This a test machine I use when I experiments on new software, and the CPU is a i7 980X in an Asus P6T motherboard with 24GB DDR3 1600MHz RAM, ordinary harddisk etc.

I have done ordinary folding with a manual installation av fah earlier, and the machine was very stabil over several weeks.

What do you think went wrong here?
 
Overclock.

As fahinstall sets Kraken up and enables "performance" scaling governor, your hardware will be stressed much more.
 
Please run the following and give me the URL's they return:
Code:
cat /etc/fstab | pastebinit
./fahdiag | pastebinit
 
Last edited:
Thanks, this was quick!

It is overclocked to 4.2GHz at the moment. I did clock it down from 4.3GHz as was the speed before, and it was stabil then. Maybe I should take it down to 4.1GHz and se what happen?

Code:
vidar@Server980X:~$ cat /etc/fstab
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
proc            /proc           proc    nodev,noexec,nosuid 0       0
# / was on /dev/sdb1 during installation
UUID=662c6df1-f9a6-4851-83ad-29b450f195d2 /               ext4    errors=remount-ro 0       1
# swap was on /dev/sdb2 during installation
UUID=870749aa-7f98-4e2f-aa4d-062f4ea30111 none            swap    sw              0       0
tmpfs  /home/vidar/fah  tmpfs  rw,uid=1000,gid=1000  0  0
vidar@Server980X:~$
 
Trying 4.1 GHz makes sense but, before you go there, run this as well, please:
Code:
fahdiag | pastebinit
if fahinstall did its job correctly, this should spit out pastebin URL containing various diagnostic information.
 
Current fahinstall script downloads and automatically installs fahdiag so it's available system-wide.

That said, drop ./ -- it's just:
Code:
fahdiag | pastebinit
 
vidar@Server980X:~$ fahdiag | pastebinit
/usr/bin/fahdiag: line 83: langouste3: command not found
md5sum: /home/vidar/fah/langouste-helper.sh: No such file or directory
ls:can not open /home/vidar/fah/langouste-helper.sh: No such file or directory
http://paste.ubuntu.com/5718958/
vidar@Server980X:~$
 
Thanks for the paste :)

I've examined diagnostic information and it actually looks fine.

When you say you were ok with 'manual' installation, I presume it was Linux as well, correct?
Which version was it?

I'd try to lower the OC a little bit more and see what happens...
 
I have clocked the memory down from 1603MHz to 150x some thing, and the CPU down to 4180MHz. It have after this been folding from 3 to 10% without error, so maybe it was more stress on the machine then before, so looking good so far.
 
Yes, Ubuntu 11.10, kraken and langouste. I did use to fold bigadv 69xx before on this machine, back in those days when that was possible.

Thanks for the paste :)

I've examined diagnostic information and it actually looks fine.

When you say you were ok with 'manual' installation, I presume it was Linux as well, correct?
Which version was it?

I'd try to lower the OC a little bit more and see what happens...

Update: The rig is stable again and now finish 2 WUs.

Big Thanks to tear and musky.;)
 
Last edited:
Then I have spent 15 minutes of to day to configure folding on ramdisk from the Xeon E5-4650 server, and it seems to be profitable. TPF for the P8104 have improved from Best before = 5:03 to Best now = 5.0. Possible that it manages to sink down to 4:5x something when it has been folding a while, so this looks good. PPD / Watt is now at 1544p. pr. Watt which I am very pleased with. It is nearly 2 times as good as my G34 6176 server that has 819p pr. Watt.


Big thanks to musky who made this possible!

Update: My SR-2 Xeon E5645 TPF improved by 30 seconds folding from ramdisk vs. regular hard drive. This was with 8105 project.

Update2: A new P8104 with a TPF of 4:58 came down on P4 E5-4650, so ramdisk was a smart configuration.

And then it is 4:57.:D
 
Last edited:
Back
Top