java xml TransformerFactory.newTransformer getting slower during tests

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

java xml TransformerFactory.newTransformer getting slower during tests

Sven Richter
I have a problem where I do not know what the root cause actually is.
I also asked already in different places without getting an answer.

I create an instance of javax.xml.transform.TransformerFactory and
directly afterwards I parse a xsltSource:


protected synchronized Transformer getTransformer(Source xsltSource)
    throws TransformerConfigurationException {

    TransformerFactory transformerFactory = TransformerFactory.newInstance();
    Transformer transformer = transformerFactory.newTransformer(xsltSource);
    return transformer;
}

I am using the com.sun.org.apache.xalan.internal.xsltc.trax.Trans
formerFactoryImpl com.sun.org.apache.xalan.internal.xsltc.trax.Trans
formerImpl classes.
Which are the ones provided by the JDK.


Now I have a test that is part of the test suite. When I run that test
standalone the code above takes around 1ms constantly.

When I run that test as part of my test suite it takes longer. This
happens as well in eclipse as with gradle. The duration rises linear
with the amount of tests that run before.

Its orders of magnitudes slower, like 10 - 1000 times, depending on
how much tests run before that specific test.

Using a cached instance of the TransfomerFactory will cut the time the
test takes in half. But the symptom remains the same.
I have been profiling it, but dont see anything suspicious, except the
the test takes longer.

My first assumption was it had directly to do with the number of
loaded classes, which it does not. Loading classes arbitrarily does
not change the time the test needs.

Next I added the saxon parser and specifically used that one, but the
symptom remains the same.

Also I figured out I have the same problem with this line of code:

SAXParserFactory saxParserFactory = SAXParserFactory.newInstance();

Any ideas what might be the cause here?

Thanks,
Sven
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: java xml TransformerFactory.newTransformer getting slower during tests

Christoffer Bruun
Hi Sven,

Have you tried reusing a single TransformerFactory to see if it is the
many TransformerFactories that somehow makes things slow ?

- I think the common pattern is to have a single factory to create all
Transforms.

Christoffer Bruun


Den 3/1/2017 kl. 10:27 AM skrev Sven Richter:

> I have a problem where I do not know what the root cause actually is.
> I also asked already in different places without getting an answer.
>
> I create an instance of javax.xml.transform.TransformerFactory and
> directly afterwards I parse a xsltSource:
>
>
> protected synchronized Transformer getTransformer(Source xsltSource)
>      throws TransformerConfigurationException {
>
>      TransformerFactory transformerFactory = TransformerFactory.newInstance();
>      Transformer transformer = transformerFactory.newTransformer(xsltSource);
>      return transformer;
> }
>
> I am using the com.sun.org.apache.xalan.internal.xsltc.trax.Trans
> formerFactoryImpl com.sun.org.apache.xalan.internal.xsltc.trax.Trans
> formerImpl classes.
> Which are the ones provided by the JDK.
>
>
> Now I have a test that is part of the test suite. When I run that test
> standalone the code above takes around 1ms constantly.
>
> When I run that test as part of my test suite it takes longer. This
> happens as well in eclipse as with gradle. The duration rises linear
> with the amount of tests that run before.
>
> Its orders of magnitudes slower, like 10 - 1000 times, depending on
> how much tests run before that specific test.
>
> Using a cached instance of the TransfomerFactory will cut the time the
> test takes in half. But the symptom remains the same.
> I have been profiling it, but dont see anything suspicious, except the
> the test takes longer.
>
> My first assumption was it had directly to do with the number of
> loaded classes, which it does not. Loading classes arbitrarily does
> not change the time the test needs.
>
> Next I added the saxon parser and specifically used that one, but the
> symptom remains the same.
>
> Also I figured out I have the same problem with this line of code:
>
> SAXParserFactory saxParserFactory = SAXParserFactory.newInstance();
>
> Any ideas what might be the cause here?
>
> Thanks,
> Sven

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: java xml TransformerFactory.newTransformer getting slower during tests

philip.miess
In reply to this post by Sven Richter
Sven,
        We were seeing a similar problem when compiling hundreds of xsltc transforms.
We saw increasing memory consumption and so blamed compiling and loading lots of transform classes.
We felt that this theory was confirmed when we found that running under Java 1.8 removed this problem.
Because java 1.8 added garbage collection of classes.
So, if you are not running Java 1.8, consider trying it.

Phil

-----Original Message-----
From: Sven Richter [mailto:[hidden email]]
Sent: Wednesday, March 01, 2017 4:28 AM
To: [hidden email]
Subject: java xml TransformerFactory.newTransformer getting slower during tests

I have a problem where I do not know what the root cause actually is.
I also asked already in different places without getting an answer.

I create an instance of javax.xml.transform.TransformerFactory and directly afterwards I parse a xsltSource:


protected synchronized Transformer getTransformer(Source xsltSource)
    throws TransformerConfigurationException {

    TransformerFactory transformerFactory = TransformerFactory.newInstance();
    Transformer transformer = transformerFactory.newTransformer(xsltSource);
    return transformer;
}

I am using the com.sun.org.apache.xalan.internal.xsltc.trax.Trans
formerFactoryImpl com.sun.org.apache.xalan.internal.xsltc.trax.Trans
formerImpl classes.
Which are the ones provided by the JDK.


Now I have a test that is part of the test suite. When I run that test standalone the code above takes around 1ms constantly.

When I run that test as part of my test suite it takes longer. This happens as well in eclipse as with gradle. The duration rises linear with the amount of tests that run before.

Its orders of magnitudes slower, like 10 - 1000 times, depending on how much tests run before that specific test.

Using a cached instance of the TransfomerFactory will cut the time the test takes in half. But the symptom remains the same.
I have been profiling it, but dont see anything suspicious, except the the test takes longer.

My first assumption was it had directly to do with the number of loaded classes, which it does not. Loading classes arbitrarily does not change the time the test needs.

Next I added the saxon parser and specifically used that one, but the symptom remains the same.

Also I figured out I have the same problem with this line of code:

SAXParserFactory saxParserFactory = SAXParserFactory.newInstance();

Any ideas what might be the cause here?

Thanks,
Sven
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: java xml TransformerFactory.newTransformer getting slower during tests

Sven Richter
Hi Philip,

thanks for the reply. Actually, in the last days I figured out that
our problems were a bit different. We are running already on JDK 8,
and memory consumption was not that much of a problem.
Instead in our algorithm we
1. Created a new instance of the TemplateFactory very often
2. Created a new instance of the SAXParserFactory very often
3. Did not cache a template that we recompiled very often too, while
it was the same.

So the solution was to create one instance of the TemplateFactory and
the SAXParserfactory, make sure they dont run multithreaded and pass
them down the call chain, as well as caching our one template and
reusing that.

Last but not least, I figured out that there is a chain that the JDK
uses to look for an implementation of the TemplateFactory and
SAXParserFactory using the ServiceLoader. Which took a really long
time in eclipse, as somehow the places, where the ServiceLoader looks
for implementations got filled up over time (you can read that in the
docs of the ServiceLoader, where this actually happens).
Passing a system property for the default implementations of these
interfaces will fix that too.

All in all our performance was improved by over 90% :-)

On Wed, Mar 8, 2017 at 10:54 PM,  <[hidden email]> wrote:

> Sven,
>         We were seeing a similar problem when compiling hundreds of xsltc transforms.
> We saw increasing memory consumption and so blamed compiling and loading lots of transform classes.
> We felt that this theory was confirmed when we found that running under Java 1.8 removed this problem.
> Because java 1.8 added garbage collection of classes.
> So, if you are not running Java 1.8, consider trying it.
>
> Phil
>
> -----Original Message-----
> From: Sven Richter [mailto:[hidden email]]
> Sent: Wednesday, March 01, 2017 4:28 AM
> To: [hidden email]
> Subject: java xml TransformerFactory.newTransformer getting slower during tests
>
> I have a problem where I do not know what the root cause actually is.
> I also asked already in different places without getting an answer.
>
> I create an instance of javax.xml.transform.TransformerFactory and directly afterwards I parse a xsltSource:
>
>
> protected synchronized Transformer getTransformer(Source xsltSource)
>     throws TransformerConfigurationException {
>
>     TransformerFactory transformerFactory = TransformerFactory.newInstance();
>     Transformer transformer = transformerFactory.newTransformer(xsltSource);
>     return transformer;
> }
>
> I am using the com.sun.org.apache.xalan.internal.xsltc.trax.Trans
> formerFactoryImpl com.sun.org.apache.xalan.internal.xsltc.trax.Trans
> formerImpl classes.
> Which are the ones provided by the JDK.
>
>
> Now I have a test that is part of the test suite. When I run that test standalone the code above takes around 1ms constantly.
>
> When I run that test as part of my test suite it takes longer. This happens as well in eclipse as with gradle. The duration rises linear with the amount of tests that run before.
>
> Its orders of magnitudes slower, like 10 - 1000 times, depending on how much tests run before that specific test.
>
> Using a cached instance of the TransfomerFactory will cut the time the test takes in half. But the symptom remains the same.
> I have been profiling it, but dont see anything suspicious, except the the test takes longer.
>
> My first assumption was it had directly to do with the number of loaded classes, which it does not. Loading classes arbitrarily does not change the time the test needs.
>
> Next I added the saxon parser and specifically used that one, but the symptom remains the same.
>
> Also I figured out I have the same problem with this line of code:
>
> SAXParserFactory saxParserFactory = SAXParserFactory.newInstance();
>
> Any ideas what might be the cause here?
>
> Thanks,
> Sven
Loading...