Sam,

  One thing in your script - you should not use the "mpiexec" launcher unless your program makes use of the MPI libraries.

  For your memory issue, I believe that you need to tell the java process to use more memory - you can set the initial and max heap sizes with the -Xms and -Xmx arguments.  So, "java -Xms32m -Xmx128m" specifies 32 MB initial heap size and 128 MB max heap size - I believe these are the defaults.  The Java documentation should have more info on these, and there may be other command line options to increase the memory available to Java.

  For using multiple CPUs to speed up your process, this needs to be built into your code to explicitly distribute work to additional CPUs.  The two main ways of doing this are using OpenMP which works with CPUs that share memory, and can use up to 2 CPUs in our cluster.  The other approach is MPI (Message Passing Interface) which can make use of CPUs on multiple nodes.  In general, both of these require that your program in C, C++, or Fortran.  There is a MPI library for Java (Google "mpi java"), but we don't have any experience using it.   In Java, you could also make use of threads, but since our nodes are only dual CPU, you're probably limited to 2 threads.  If you have questions about doing MPI or OpenMP using C, C++, or Fortran, we can help with that.

Dave

-----Original Message-----
From: Arockiasamy, Savarimuthu Mr. 
Sent: Tuesday, June 09, 2009 12:54 PM
To: Woods, David M. Dr.; 'Research Computing Support'
Subject: RE: Using Redhawk

Dr.Dave,

I tested my program on the cluster computer with a small input file. It worked. But, when I tried to run that with the actual file, it fails with the Java heap space error. I used "bigmem" to use one of the node that has 10GB memory. But, it still fails with the memory error. I am not sure if I am doing anything wrong in the batch job. Could you please take a look at the batch job below? Also, I would like to know how to use multiple CPU/Processors to speed up the process.

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
mpiexec: Warning: task 0 exited before completing MPI startup.

[arockis@mulnx37 ~]$ more launch.sh
#!/bin/bash -l
#PBS -N SemanticSimilarityTest
#PBS -l nodes=1:bigmem:ppn=1
#PBS -l walltime=2:00:00
#PBS -m abe
#PBS -M [log in to unmask]
#PBS -V
mpiexec java SemanticSimilarityTest SNMI_SKOS.owl CUI PC conceptPairs.txt SNMI_SemSimilarity
[arockis@mulnx37 ~]$

Thanks,
Sam.

-----Original Message-----
From: Arockiasamy, Savarimuthu Mr. 
Sent: Monday, June 08, 2009 11:13 AM
To: Woods, David M. Dr.; Research Computing Support
Subject: RE: Using Redhawk

Thanks so much, Dr.Dave. I will read over this document.

Sam.

-----Original Message-----
From: Woods, David M. Dr. 
Sent: Monday, June 08, 2009 9:11 AM
To: Research Computing Support; Arockiasamy, Savarimuthu Mr.
Subject: RE: Using Redhawk

Hi,

  To run this code on the cluster, the first thing you will need to do is modify it to eliminate the use of the GUI interface.  The cluster supports batch processing, so your input will need to be provided on the command line or read from an parameter file.  You need to be able to run your code from a Linux command prompt.

  Once your code is setup, copy it and any required files to the cluster and setup a batch job script to run the program.  The cluster has 256 CPUs, so you can run multiple analyses at the same time on different files or with different parameters, but you will need to ensure that the different analyses aren't writing to the same files, etc.

  I've attached a document with info on using the batch scheduling system on the cluster.  

  Hopefully this gets you started.  I'm sure you'll have more questions.

Dave


David Woods, Ph.D
Assistant Director for Research Computing
357 Gaskill Hall
Miami University
(513) 529-1857




-----Original Message-----
From: Research Computing Support [mailto:[log in to unmask]] On Behalf Of Arockiasamy, Savarimuthu Mr.
Sent: Sunday, June 07, 2009 9:50 PM
To: [log in to unmask]
Subject: Re: Using Redhawk

I am trying to run a Java program that reads three input files and do some complex processing. At the end of the process, the program creates a report. This program runs forever on my laptop. 

The program has a GUI interface where I have to enter some values. I want to run this program for different input files and different values on the GUI. Basically I want to run the same program for different input files and input values(entered on the GUI) at the same time. Could you please guide me how to do this in the cluster computing environment? 

Thanks for your help,
Sam.



-----Original Message-----
From: Woods, David M. Dr. 
Sent: Thursday, June 04, 2009 9:43 AM
To: Cross, Valerie Dr.
Cc: Arockiasamy, Savarimuthu Mr.
Subject: RE: Using Redhawk

Valerie,

  Thanks for the update.  

  Sam - let us know what we can do to help you.  If you send e-mail to [log in to unmask] it will go to the entire group and get the quickest response.

Dave


-----Original Message-----
From: Cross, Valerie Dr. 
Sent: Thursday, June 04, 2009 1:32 AM
To: Woods, David M. Dr.
Cc: Arockiasamy, Savarimuthu Mr.
Subject: Using Redhawk


Hi Dave,

I have my one student Xinran learning Python.  She is doing well and so far we do not need any help on the python programming.   We got very busy and my other grad student Zhisong Pei started his job early and left early so the VTK stuff for now is on the back burner.  Hopefully, we will get back to it.  But for right now,  I am concentrating on working with my other graduate student Sam Arockisamy whom I have copied on this email.   We are trying to run some experiments that are taking forever on his laptop computer.  

I have set up an account for him on the redhawk cluster computer and he will be working on running the same program with different inputs.    We would like to partition the input data and run it using multiple processors to try to get results out ASAP because we are working on a conference paper which is due June 15.   Anyway,  I just wanted to let you know that he may contact you for getting help from the Research Computing Support Group on using the cluster computer if has problems or questions.  

Thanks for your help as always.   I really appreciate it.

Valerie