Center for Computational Molecular Science and Technology Georgia Institute of Technology Center for Computational Molecular Science and Technology School of Chemistry and Biochemistry

CCMST Weekly News, February 3 2012

February 3, 2012 3:12 pm EST

1. Announcements
2. Tip of the Week

ANNOUNCEMENTS

AACP Seminar

February 6, 2012 8:00 PM – 9:00 PM
Emory University, 316 Atwood Hall
Prof. Christine Payne
Georgia Tech

School Colloquium

February 9, 2012 4:00 PM – 5:00 PM
MoSE G011
Prof. Herman Sintim, University of Maryland
Disrupting Bacterial Networks with Synthetic Small Molecules

Green Computing Web Site

We are proud to announce that the Green Computing web site is now on-line. The web site can be visited at the address http://www.greencomputing.gatech.edu/ .

The site presents content related to the "Green Computing for Green Energy" NSF-CRIF grant proposal that led to the purchase of the Green Gate computer cluster.

TIP OF THE WEEK

By Massimo

Backup Scripts

Although user data on the CCMST clusters are backed up daily, it is a good idea to periodically compress and backup old data, to keep disk usage low. Once the data are compressed, it is easy to transfer them to a different storage.

Here follow sample scripts for Moab/Torque that can be used to compress directories of user data for long term storage. These scripts can be submitted to the batch queue as normal jobs. They work using the local scratch space of the compute node as temporary storage, thus speeding up the compression process and easing the load on the NFS server. The first script uses the zip command, in case the data is to be transferred to a Windows computer, the second script uses the tar command, suitable for Unix systems.

  • Zip Script
    #!/bin/bash
    #
    # this command will zip the content of a directory, place the zip file
    # in the directory ~/zipped and delete the original directory.
    #
    # to use it, replace the name "apoa1" with the directory you want to zip.
    # the script assumes the directory to zip is under the user home directory (~).
    # if the directory is somewhere else, just modify the path to point to
    # the directory you want to zip.
    #
    # the script works like this:
    #
    # 1) the zip file is created under the user scratch directory (/scratch/${USER})
    #
    # 2) the content of the zip file is verified (option -T of the zip command)
    #
    # 3) if the verification is successfull, the original directory is
    # removed (option -m of the zip command)
    #
    # 4) the zipped file is copied under ~/zipped
    #
    # If you do not want to delete the original directory (for later manual
    # deleting) ndo not use the -m option.
    #
    ##############################################################################
    #
    # script starts here. In the following, substitute "apoa1" with the name of
    # the directory you want to zip
    #
    ##############################################################################
    #
    # job name (enter your job name here)
    #
    #PBS -N zip_apoa1
    #PBS -J oe
    #
    # requests 1 GB of memory
    #
    #PBS -l pmem=1000mb
    #
    #
    #
    #

    tozip=apoa1
    scrdir=/scratch/${USER}
    zipdir=~/zipped

    mkdir -p ${scrdir} # create scratch directory (if it does not exist)
    mkdir -p ${zipdir} # create storage directory (if it does not exist)
    cd ${scrdir} # cd to scratch directory
    zip -T -m -r ${tozip}.zip ~/${tozip} # zip, verify and delete original
    cp ${tozip}.zip ${zipdir} # copy zip file to storage directory
    rm ${tozip}.zip # remove zip file from scratch
  • Tar Script
  • #!/bin/bash
    #
    # this command will create a compressed tarball of the content of a directory, place the tar file
    # in the directory ~/tarballs and delete the original directory.
    #
    # to use it, replace the name "apoa1" with the directory you want to compress
    # the script assumes the directory to compress is under the user home directory (~).
    # if the directory is somewhere else, just modify the path to it
    #
    # the script works like this:
    #
    # 1) the tar file is created under the user scratch directory (/scratch/${USER})
    #
    # 2) the content of the tar file is verified (option -W of the tar command)
    #
    # 3) if the verification is successfull, the original directory is
    # removed (option --remove-files of the tar command)
    #
    # 4) the compressed file is copied under ~/tarballs
    #
    # If you do not want to delete the original directory (for later manual
    # deleting) do not use the --remove-files option.
    #
    ##############################################################################
    #
    # script starts here. In the following, substitute "apoa1" with the name of
    # the directory you want to compress
    #
    ##############################################################################
    #
    # job name (enter your job name here)
    #
    #PBS -N tar_apoa1
    #PBS -J oe
    #
    # requests 1 GB of memory
    #
    #PBS -l pmem=1000mb
    #
    totar=apoa1
    scrdir=/scratch/${USER}
    tardir=~/tarballs

    mkdir -p ${scrdir} # create scratch directory (if it does not exist)
    mkdir -p ${tardir} # create storage directory (if it does not exist)
    cd ${scrdir} # cd to scratch directory
    tar -W --remove-files -cvzf ${totar}.tar.gz ~/${totar} # tar, verify and delete original
    cp ${totar}.tar.gz ${tardir} # copy tar file to storage directory
    rm ${totar}.tar.gz # remove tar file from scratch

Do you have usage tips that you want to share with the other CCMST users? Please send them to Massimo (massimo.malagoli@chemistry.gatech.edu) for inclusion in the Tip of the Week section.