External workflows hanging in Teamsite.

System

We have an issue with our workflow process and are hoping someone else has experience with it. We have an open support ticket number is 1214954 currently 12 days old on our production system.

The Issue:
We are able to create a dcr but when submitting to staging the external workflow hangs. This was working; data was being sent to the database and files were being being deployed to the proper locations. This process then stopped working, we have verified that the files involved have not been altered and there have been no changes to the environement.

Our Process:
The direct submit external workflow itself calls the perl deployment script, which calls the Opendeploy deployment script which calls the datadeploy deployment script. This process again worked for two weeks without issue and then the the external workflow hung in TeamSite there are no errors generated and nothing appears (job number) is displayed in the workflow tab located in the professional view of TeamSite.

It is interesting becuase the file submited as part of the workflow does end up in staging, but the deployment piece does not appear to run.
We cant tell exactly where it hangs... But the submit to staging does occur and the deployments do not run at all (hence there are no OD logs for the deployments...)

We have another workflow, the standard submit to staging from a workarea, there are no issues with this workflow. Our users are able to submit content and all other scheduled daily deployments are running sucessfully.

In the process of trying to manually cancel all hung workflow tasks, TeamSite itself hung and went down and had to be restarted.

While working with support we removed the workflow directory from Backing Store and restarted TeamSite but it had no effect resolving this issue.

Our Platform:
Solaris 8
TeamSite version 6.1 SP 1
Open & Data Deploy 6.0

There is nothing unusual running in the background, no big batch jobs or deployments. We have a small by most company's standards TeamSite implementation. We would really appreciate any help that can help to remedy this issue.

thks .c & .e


We have provided thus far to Interwoven support copies of the below files listed:
debug view of workflow
direct_submit_deploy.wft
available_templates.cfg
od_deploy.ipl

iwtrace log
iwjoberrors log
iwevents log

<iwhome>/local/logs/iwui * (please get all logs in this directory (full logs).
<iw-home>/local/logs/* (please send us the latest logs out of this directory.
<iw-home>/iw-webd/logs/* please get all logs in this directory (full logs).
<iw-home>/servletd/logs/*