another patch for multi-process issues (was: patch for intermittent Ant failures)

[Vadim Nasardinov <vadimn@xxxxxxxxxx>]

When I tried to run CAROL's unit tests on the carol_1_8_5_br branch,
as suggested in [1], I ran into the following problem.  (Matter of
fact, I'd seen this happen before, but didn't have the time to get to
the bottom of this until today.)

I saw a lot of errors like this [2]:

 | report:
 | [junitreport] [Fatal Error]
 |   
TEST-org.objectweb.carol.jtests.conform.basic.clients.MultiProtocolTests.xml:830:1:
 |   Content is not allowed in trailing section.
 | [junitreport] The file
 |   
/var/vadim/scratch/ow/carol-1_8_5_br/output/dist/test/results/multi_suniiop_jeremie/TEST-org.objectweb.carol.jtests.conform.basic.clients.MultiProtocolTests.xml
 |   is not a valid XML document. It is possibly corrupted.
 | [junitreport] Transform time: 363ms


So, this says the XML file appears corrupted at line 830.  Sure
enough, if you look at the offending file [3], you'll notice that
there is something funny going on around line 830:


| $ cat -n 
TEST-org.objectweb.carol.jtests.conform.basic.clients.MultiProtocolTests.xml 
| head -n 832 | tail -n 7
|    826                at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:656)
|    827                at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:558)
|    828        ]]></system-err>
|    829        </testsuite>
|    830        esult.runProtected(TestResult.java:124)
|    831                at junit.framework.TestResult.run(TestResult.java:109)
|    832                at junit.framework.TestCase.run(TestCase.java:131)


There are two closing </testsuite> tags.  One at line 830, and the
other one at the end of the file at line 1032:

| $ cat -n 
TEST-org.objectweb.carol.jtests.conform.basic.clients.MultiProtocolTests.xml 
| tail -n 3
|   1030                at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:558)
|   1031        ]]></system-err>
|   1032        </testsuite>

The resulting XML is obviously invalid, which prevents the
<junitreport> task from being able to generate an HTML version of the
report.  (Incidentally, this also prevents XML-capable browsers like
Mozilla from displaying the file correctly.  You'll need to save it to
disk and open it up in a text editor in order to see the file
contents.)

The fact the markup is corrupted leads JunitReport to not finding any
test results at all, as shown in [4]:

   +---------------------------------------------------+
   |            Suite            |Tests|Failures|Errors|
   |-----------------------------+-----+--------+------|
   |multi_sunjrmp1.2_suniiop.nons|4    |0       |0     |
   |-----------------------------+-----+--------+------|
   |multi_sunjrmp1.1_suniiop.nons|4    |0       |0     |
   |-----------------------------+-----+--------+------|
   |multi_suniiop_sunjrmp1.2.nons|4    |0       |0     |
   |-----------------------------+-----+--------+------|
   |multi_suniiop_sunjrmp1.1.nons|4    |0       |0     |
   |-----------------------------+-----+--------+------|
   |multi_suniiop_jeremie.nons   |0    |0       |0     |
   |-----------------------------+-----+--------+------|
   |multi_sunjrmp1.2_suniiop     |0    |0       |0     |
   |-----------------------------+-----+--------+------|
   |multi_sunjrmp1.1_suniiop     |0    |0       |0     |
   |-----------------------------+-----+--------+------|
   |multi_suniiop_sunjrmp1.2     |4    |0       |0     |
   |-----------------------------+-----+--------+------|
   |multi_suniiop_sunjrmp1.1     |4    |0       |0     |
   |-----------------------------+-----+--------+------|
   |multi_suniiop_jeremie        |0    |0       |0     |
   |-----------------------------+-----+--------+------|
   |sunjrmp1.2.nons              |4    |0       |0     |
   |-----------------------------+-----+--------+------|
   |sunjrmp1.1.nons              |4    |0       |0     |
   |-----------------------------+-----+--------+------|
   |suniiop.nons                 |4    |0       |0     |
   |-----------------------------+-----+--------+------|
   |jeremie.nons                 |4    |0       |0     |
   |-----------------------------+-----+--------+------|
   |sunjrmp1.2                   |4    |0       |0     |
   |-----------------------------+-----+--------+------|
   |sunjrmp1.1                   |4    |0       |0     |
   |-----------------------------+-----+--------+------|
   |suniiop                      |4    |0       |0     |
   |-----------------------------+-----+--------+------|
   |jeremie                      |4    |0       |0     |
   +---------------------------------------------------+

Note, for example, that 0 tests are reported to have been run for the
"multi_suniiop_jeremie.nons" suite.  In reality, 4 tests were run by
each of the two independently running JUnit processes.

This happens because of the way CAROL's runs "multi" tests.  If you
take a look at, for example,
carol/test/conform/generic/carol_double.xml,v1.11, you'll notice that
the "launch.test.double" target launches _two_ JUnit processes in
parallel.  They both output test results to the same file.  So, now
you have two separate processes stepping on each other's toes by
writing to the same file.  The output gets corrupted.

The attached patch fixes this problem by making these two JUnit
processes use two different files for outputting test results.

This produces reports such as those shown in [5].


Footnotes

 1. http://mail-archive.objectweb.org/carol/2004-12/msg00015.html
 2. 
http://people.redhat.com/vadimn/scratch/ow/carol-coverage/1_8_5_br/2004-12-20/corrupted-results/test-results.txt
 3. 
http://people.redhat.com/vadimn/scratch/ow/carol-coverage/1_8_5_br/2004-12-20/corrupted-results/multi_suniiop_jeremie/TEST-org.objectweb.carol.jtests.conform.basic.clients.MultiProtocolTests.xml
 4. 
http://people.redhat.com/vadimn/scratch/ow/carol-coverage/1_8_5_br/2004-12-20/corrupted-results/
 5. http://mail-archive.objectweb.org/carol/2004-12/msg00017.html

Index: test/conform/generic/carol_double.xml
===================================================================
RCS file: /cvsroot/carol/carol/test/conform/generic/carol_double.xml,v
retrieving revision 1.11
diff -u -r1.11 carol_double.xml
--- test/conform/generic/carol_double.xml	9 Feb 2004 11:00:24 -0000	1.11
+++ test/conform/generic/carol_double.xml	20 Dec 2004 22:02:15 -0000
@@ -23,6 +23,8 @@
 
   <!-- test result location -->
   <property name="report.dir"  value="${test.dist.results}/${test.name}"/>
+  <property name="parallel.report.dir1"  value="${report.dir}/parallel1"/>
+  <property name="parallel.report.dir2"  value="${report.dir}/parallel2"/>
 
   <!-- test tmp location -->
   <property name="test.tmp.client1"               value="${test.tmp}/client/1"/>
@@ -137,7 +139,7 @@
   </target>
 
   <!-- test launching --> 
-  <!-- those tests are launch with a parallele task -->
+  <!-- those tests are launch with a parallel task -->
   <target name="launch.test.double">
     
     <condition property="is.jvm.1.4">
@@ -167,16 +169,14 @@
 
       <sequential>
 	<sleep seconds="10"/>
-	<delete dir="${report.dir}" />
-	<mkdir dir="${report.dir}" />
-	<mkdir dir="${report.dir}/html" />
+	<mkdir dir="${parallel.report.dir1}" />
 	<junit printsummary="yes" 
 	  haltonfailure="no"
 	  haltonerror="no"
 	  fork="yes" >
 	  <classpath refid="test.client.classpath1" />
 	  <formatter type="xml"/>
-	  <batchtest fork="yes" todir="${report.dir}">
+	  <batchtest fork="yes" todir="${parallel.report.dir1}">
 	    <fileset dir="${test.src}">
 	      <include name="${test.client.pkg1}/*Tests.java" />
 	    </fileset>
@@ -186,15 +186,14 @@
 
       <sequential>
 	<sleep seconds="10"/>
-	<mkdir dir="${report.dir}" />
-	<mkdir dir="${report.dir}/html" />
+	<mkdir dir="${parallel.report.dir2}" />
 	<junit printsummary="yes" 
 	  haltonfailure="no"
 	  haltonerror="no"
 	  fork="yes" >
 	  <classpath refid="test.client.classpath2" />
 	  <formatter type="xml"/>
-	  <batchtest fork="yes" todir="${report.dir}">
+	  <batchtest fork="yes" todir="${parallel.report.dir2}">
 	    <fileset dir="${test.src}">
 	      <include name="${test.client.pkg2}/*Tests.java" />
 	    </fileset>
@@ -208,11 +207,21 @@
   </target> 
 
   <target name="report" if="is.jvm.1.4" >
-    <junitreport todir="${report.dir}">
-      <fileset dir="${report.dir}">
+    <mkdir dir="${parallel.report.dir1}/html" />
+
+    <junitreport todir="${parallel.report.dir1}">
+      <fileset dir="${parallel.report.dir1}">
+	<include name="TEST-*.xml"/>
+      </fileset>
+      <report format="frames" todir="${parallel.report.dir1}/html"/>
+    </junitreport>
+
+    <mkdir dir="${parallel.report.dir2}" />
+    <junitreport todir="${parallel.report.dir2}">
+      <fileset dir="${parallel.report.dir2}">
 	<include name="TEST-*.xml"/>
       </fileset>
-      <report format="frames" todir="${report.dir}/html"/>
+      <report format="frames" todir="${parallel.report.dir2}/html"/>
     </junitreport>
   </target>
 
Index: test/conform/generic/carol_double_nons.xml
===================================================================
RCS file: /cvsroot/carol/carol/test/conform/generic/carol_double_nons.xml,v
retrieving revision 1.4
diff -u -r1.4 carol_double_nons.xml
--- test/conform/generic/carol_double_nons.xml	9 Feb 2004 11:00:24 -0000	1.4
+++ test/conform/generic/carol_double_nons.xml	20 Dec 2004 22:02:16 -0000
@@ -21,6 +21,9 @@
 
   <!-- test result location -->
   <property name="report.dir"  value="${test.dist.results}/${test.name}"/>
+  <property name="parallel.report.dir1"  value="${report.dir}/parallel1"/>
+  <property name="parallel.report.dir2"  value="${report.dir}/parallel2"/>
+
 
   <!-- test tmp location -->
   <property name="test.tmp.client1"               value="${test.tmp}/client/1"/>
@@ -122,16 +125,14 @@
 
       <sequential>
 	<sleep seconds="10"/>
-	<delete dir="${report.dir}" />
-	<mkdir dir="${report.dir}" />
-	<mkdir dir="${report.dir}/html" />
+	<mkdir dir="${parallel.report.dir1}" />
 	<junit printsummary="yes" 
 	  haltonfailure="no"
 	  haltonerror="no"
 	  fork="yes" >
 	  <classpath refid="test.client.classpath1" />
 	  <formatter type="xml"/>
-	  <batchtest fork="yes" todir="${report.dir}">
+	  <batchtest fork="yes" todir="${parallel.report.dir1}">
 	    <fileset dir="${test.src}">
 	      <include name="${test.client.pkg1}/*Tests.java" />
 	    </fileset>
@@ -141,15 +142,14 @@
 
       <sequential>
 	<sleep seconds="10"/>
-	<mkdir dir="${report.dir}" />
-	<mkdir dir="${report.dir}/html" />
+	<mkdir dir="${parallel.report.dir2}" />
 	<junit printsummary="yes" 
 	  haltonfailure="no"
 	  haltonerror="no"
 	  fork="yes" >
 	  <classpath refid="test.client.classpath2" />
 	  <formatter type="xml"/>
-	  <batchtest fork="yes" todir="${report.dir}">
+	  <batchtest fork="yes" todir="${parallel.report.dir2}">
 	    <fileset dir="${test.src}">
 	      <include name="${test.client.pkg2}/*Tests.java" />
 	    </fileset>
@@ -163,11 +163,21 @@
   </target> 
 
   <target name="report" if="is.jvm.1.4" >
-    <junitreport todir="${report.dir}">
-      <fileset dir="${report.dir}">
+    <mkdir dir="${parallel.report.dir1}/html" />
+
+    <junitreport todir="${parallel.report.dir1}">
+      <fileset dir="${parallel.report.dir1}">
+	<include name="TEST-*.xml"/>
+      </fileset>
+      <report format="frames" todir="${parallel.report.dir1}/html"/>
+    </junitreport>
+
+    <mkdir dir="${parallel.report.dir2}" />
+    <junitreport todir="${parallel.report.dir2}">
+      <fileset dir="${parallel.report.dir2}">
 	<include name="TEST-*.xml"/>
       </fileset>
-      <report format="frames" todir="${report.dir}/html"/>
+      <report format="frames" todir="${parallel.report.dir2}/html"/>
     </junitreport>
   </target>