Restrict CPU usage by java8 parallel stream/High CPU usage by Java 8 parallel stream
I was testing the performance of java 8 parallel stream api by creating a simple loop and adding elements in an array.
I am getting a huge performance boost over non parallel one.
But when I am checking my task manager I am seeing an uncontrolled CPU usage, it eats up all my CPU during that period.
here is my sample code,
public static void withIfElse(boolean sorted) {
int[] arr = new int[32768];
Random rand = new Random();
for(int i = 0;i < arr.length;i++)
arr[i] = rand.nextInt(256);
if(sorted)
Arrays.parallelSort(arr);
long nst = System.currentTimeMillis();
long sum = 0;
for(int j=0;j<100000;j++) {
sum += Arrays.stream(arr).parallel().filter(m->m>=128).reduce((m,n)-> m+n).getAsInt();
}
System.out.printf("Time taken for boolean %b is %d ms. \n",sorted,(System.currentTimeMillis()-nst));
}
I am able to achieve almost 2x to 8x performance boost.
But ,
Cpu usage is uncontrolled. Is there any way to force java to only 2 cores ?
I have also tried setting
System.setProperty("java.util.concurrent.ForkJoinPool.common.parallelism", "1");
But still CPU usage is high.
Also is it recommended to use parallel stream?, because suppose in my application I have 10 user threads, or a web application where each request is a thread, in that case if I start a parallel stream on multiple threads which eventually eat all my cpu, there will be so many context switching.
2 answers
-
answered 2017-06-17 18:42
Joe C
Parallel streams run, by default, in the system-wide ForkJoinPool, which gives one thread for each core available on your machine.
If you want to change this behaviour, you can run your parallel stream in a ForkJoinPool of your choosing:
ForkJoinPool fjp = new ForkJoinPool(2); ForkJoinTask<Long> sumFuture = fjp.submit(() -> { long sum = 0; for(int j=0;j<100000;j++) { sum += Arrays.stream(arr).parallel().filter(m->m>=128).reduce((m,n)-> m+n).getAsInt(); } return sum; });
-
answered 2017-06-17 18:42
Eugene
My 0.02$:
huge performance boost over non parallel one
. huge is probably a bit too much here. And there might be good reasons for this - first, the way you test. It is so critical that there has been a question with over 500 up-votes especially for this: micro-benchmark. You might think you have good tests written, but jvm could have different plans for you.For moderate data, indeed a sequential stream might be faster than a parallel one. It's a lot more work that has to be done by a parallel stream. But without actual measurements, it's close to impossible to say; but here are recommendations from the best in the field, like this.
When you say that the you want to limit java to some cores - the plain answer is that you can't; from java code that is. It has to be done from the
OS
, if it doable at all. What you have enable there viajava.util.concurrent.ForkJoinPool.common.parallelism
is limiting the threads, not the cores.And a single thread can be split over multiple cpus - it depends on the Operating system itself.