I have a filesystem called "datafiles" that occasionally has files with the structure "upload.0002" or "upload.0003", etc.. I need to create a Korn script that can be run via CRON that will do the following:

1. Check in my "datafiles" directory for any files that have those names.
2. Insure that any files found with that naming structure have completely uploaded.
3. Look at the extension that the file has (a.k.a. "0002", "0003", "0004", etc.) and capture it as a variable.
4. Re-name the file to the "str####.asc" structure (i.e. upload.0002 would become str0002.asc)
4. Run a command called "prep" followed by the correct variable for that file, so "str0002.asc" would have "/prep 0002" run for it and "str0003.asc" would have "/prep 0003" run for it...
5. Wait 300 seconds and repeat, until the process is killed by CRON

Something like:

processFile ()
{
local fileName=$1
local fileExtension=$2
# doStuffWithFile
}

# Main Processing

for fileName in uploadDirectory/upload.[0-9][0-9][0-9][0-9]
do
fileExtension=${fileName#*.}
fileTest=/tmp/$fileName.$(date +%d%m%y_%H%M%S)
cp -f $fileName $fileTest
sleep 60
test "$fileName" != "$(find $fileName -newer $fileTest)" > /dev/null && processFile fileName fileExtension
done


...should check for the file and make sure it is done processing (thanks Damian...), but how to get the script to capture the extension and run arguments against a file using that extension as a variable I'm puzzled?

Something like this ?

After placing the following in usr/bin and running it from CRON I'm getting the error:

cp: /tmp//datafiles/upload.0100.090304_100854: A file or directory in the path name does not exist.
find: 0652-015 Cannot access file /tmp//datafiles/upload.[0-9][0-9][
0-9][0-9].090304_100754.
mv: 0653-401 Cannot rename fileName to strfileExtension.asc:
A file or directory in the path name does not exist.

...here is the modified script:

!/bin/ksh

processFile ()
{
local fileName=$1
local fileExtension=$2
local fileNewName="str${fileExtension}.asc"
mv $fileName $fileNewName # Add status check
cd /datafiles
prep /datafiles/$fileNewName $fileExtension
}

# Main Processing

while :
do
for fileName in /datafiles/upload.[0-9][0-9][0-9][0-9]
do
fileExtension=${fileName#*.}
fileTest=/tmp/$fileName.$(date +%d%m%y_%H%M%S)
cp -f $fileName $fileTest
sleep 30
test "$fileName" != "$(find $fileName -newer $fileTest)" > /dev/null && processFile fileName fileExtension
done
sleep 300
done


...it looks like the path to the filename is interfering with the test process for a "new" file?

Got it working with the following:

#!/bin/ksh

processFile ()
{
local fileName=$1
local fileExtension=$2
local fileNewName="/datafiles/str${fileExtension}.asc"
mv $fileName $fileNewName # Add status check
cd /datafiles
chmod 667 $fileNewName
prep $fileNewName $fileExtension
}

# Main Processing

while :
do
for fileName in /datafiles/upload.[0-9][0-9][0-9][0-9]
do
fileExtension=${fileName#*.}
fileTest=/tmp/$(basename $fileName).$(date +%d%m%y_%H%M%S)
cp -f $fileName $fileTest
sleep 10
test "$fileName" != "$(find $fileName -newer $fileTest)" > /dev/null && processFile $fileName $fileExtension
done
sleep 60
done

...but it still has a couple of problems:

1. When I leave in the "sleep 10" command inside the loop, it sometimes processes new files incorrectly. If someone is placing a new "upload.####" file in the /datafiles directory during that 10 second pause then the script creates a /tmp/upload.[0-9][0-9][0-9][0-9]."date"."time" file with the partially uploaded file.

2. If you remove the "sleep 10" command it no longer creates the /tmp/upload.[0-9][0-9][0-9][0-9]."date"."time" files however it ends up processing some files that are partially uploaded to the directory.

...the script needs to verify that the file is not still uploading, then process it accordingly; I need this to happen while other "upload.####" files are being created simultaneously?