Restrict CPU usage by java8 parallel stream/High CPU usage by Java 8 parallel stream

2017-06-17 18:42 RBanerjee imported from Stackoverflow

I was testing the performance of java 8 parallel stream api by creating a simple loop and adding elements in an array.

I am getting a huge performance boost over non parallel one.

But when I am checking my task manager I am seeing an uncontrolled CPU usage, it eats up all my CPU during that period.

here is my sample code,

    public static void withIfElse(boolean sorted) {

    int[] arr = new int[32768];
    Random rand = new Random();
    for(int i = 0;i < arr.length;i++)
        arr[i] = rand.nextInt(256);
    if(sorted)
        Arrays.parallelSort(arr);

    long nst = System.currentTimeMillis();
    long sum = 0;
    for(int j=0;j<100000;j++) {
        sum += Arrays.stream(arr).parallel().filter(m->m>=128).reduce((m,n)-> m+n).getAsInt();
    }

    System.out.printf("Time taken for boolean %b is %d ms. \n",sorted,(System.currentTimeMillis()-nst));

}

I am able to achieve almost 2x to 8x performance boost.

But ,

Cpu usage is uncontrolled. Is there any way to force java to only 2 cores ?

I have also tried setting

System.setProperty("java.util.concurrent.ForkJoinPool.common.parallelism", "1");

But still CPU usage is high.

Also is it recommended to use parallel stream?, because suppose in my application I have 10 user threads, or a web application where each request is a thread, in that case if I start a parallel stream on multiple threads which eventually eat all my cpu, there will be so many context switching.

2 answers

answered 2017-06-17 18:42 Joe C

Parallel streams run, by default, in the system-wide ForkJoinPool, which gives one thread for each core available on your machine.

If you want to change this behaviour, you can run your parallel stream in a ForkJoinPool of your choosing:

    ForkJoinPool fjp = new ForkJoinPool(2);
    ForkJoinTask<Long> sumFuture = fjp.submit(() -> {
        long sum = 0;
        for(int j=0;j<100000;j++) {
            sum += Arrays.stream(arr).parallel().filter(m->m>=128).reduce((m,n)-> m+n).getAsInt();
        }
        return sum;
    });

answered 2017-06-17 18:42 Eugene
My 0.02$: huge performance boost over non parallel one. huge is probably a bit too much here. And there might be good reasons for this - first, the way you test. It is so critical that there has been a question with over 500 up-votes especially for this: micro-benchmark. You might think you have good tests written, but jvm could have different plans for you.

For moderate data, indeed a sequential stream might be faster than a parallel one. It's a lot more work that has to be done by a parallel stream. But without actual measurements, it's close to impossible to say; but here are recommendations from the best in the field, like this.

When you say that the you want to limit java to some cores - the plain answer is that you can't; from java code that is. It has to be done from the OS, if it doable at all. What you have enable there via java.util.concurrent.ForkJoinPool.common.parallelism is limiting the threads, not the cores.

And a single thread can be split over multiple cpus - it depends on the Operating system itself.

See also questions close to this topic

HashMap Rehashing on the basis of Size

I am having one question that rehashing in HashMap always occurs when size reaches loadFactor*capacity+1. Now Suppose Initially there are 16 Buckets, and I am inserting Object which has a bad implementation of hashCode means returning same value i.e. 1. Now at this point even after insertion of 13 elements, rehashing will take places but if you see then all elements are in one bucket in form of LinkedList and other 15 buckets are empty.So why Rehashing done on the basis of Size and Not Capacity. In HashMap, there is a re-calculation of hashCode using out provided HashCode but that will also return the same value.
How to make corner tabs centered in TabLayout?

I'm looking for solutions to make a tablayout where the Last that is scrolled to appears at the center of the area, instead of sitting at the side.

this is the look i'm trying to go for
Retrofit Autoconfiguration in Spring
I am currently writing a Spring Boot autoconfiguration for Retrofit 2. What I am trying to do is to write some sort of an interface builder that is able instantiate an interface that is annotated with some annotation for autowiring just like Spring Data does it with repositories. As I cannot find any resources on how to do this (or if it can even be done with Spring), I would like to ask for your thoughts on that. Below is for an interface that I would like to instantiate.
My replacement for @Repository is @Retrofit the rest is just "ordinary" code you would write for any Retrofit repository.
```
@Retrofit
public interface Api {

    @GET("usernames")
    String[] getUsernames();
}
```
Java : choice between Optional and CompletableFuture
Say I have a value which will be computed at a later time and its computed result may be null. I want to express this rule in a consistant API (interface method).

I can't figure out the best option between (simplified for clarity) :
```
    CompletableFuture<String> fl = CompletableFuture.completedFuture(value);
    System.out.println("fl = " + fl.getNow(null));
```
and
```
    Optional<String> ol = Optional.ofNullable(value);
    System.out.println("ol.get() = " + ol.orElse(null));
```
1 clearly states to the API consumer that fl will be known later, but it is unclear if it may return null

2 is obviously optional, but not future bound. plus Optional here is not used as a return value but as a direct API marker

3 CompletableFuture of an T Optional seems.. fishy

4 Does annotation with @Nullable look sufficient for the person reading the code ?

5 Project Reactor has a nice Mono but I'd like to stay within the JDK itself

Which one would be the best route ?

Java stream is confusing. Giving different result while foreach on stream and with out using foreach on stream

while working on java8 steam I'm facing some issue.can anyone help me why its working different in this scenario?

public class OrderInInersion { 
static int i = 1;
public static void main(String[] args) {
    List<Image> list = getListOfImges();

    Stream<Image> stream=list.stream().sorted((Image1, Image2) -> Image1.creationOn.compareTo(Image2.creationOn)).map(img -> {
        img.id=getInt();
        return img;
    });
    stream.forEach(System.out::print);
    System.out.printf("\n");
    list.forEach(System.out::print);
}
public static int getInt() {
    return i++;
}
public static List<Image> getListOfImges() {
    List<Image> list = new LinkedList<Image>();
    list.add(new Image("rose", new Date(System.currentTimeMillis() - 400000)));
    list.add(new Image("white", new Date(System.currentTimeMillis() - 300000)));
    list.add(new Image("lilly", new Date(System.currentTimeMillis() - 600000)));
    list.add(new Image("sunflower", new Date(System.currentTimeMillis() - 900000)));
    list.add(new Image("yellow", new Date(System.currentTimeMillis() - 100000)));
    return list;
}
}
class Image {
 String name;
  int id;
  Date creationOn;
   public Image(String name, Date creationOn) {
    this.name = name;
    this.creationOn = creationOn;
    }
    public String toString() {
        return "Image [name=" + name + ", id=" + id + "]";
    }}

the output of above program.

Image [name=sunflower, id=1]Image [name=lilly, id=2]Image [name=rose, id=3]Image [name=white, id=4]Image [name=yellow, id=5]
Image [name=rose, id=3]Image [name=white, id=4]Image [name=lilly, id=2]Image [name=sunflower, id=1]Image [name=yellow, id=5]

the problem is while commenting the below line in the program. the output is coming differently.

        //stream.forEach(System.out::print);

output of program

Image [name=rose, id=0]Image [name=white, id=0]Image [name=lilly, id=0]Image [name=sunflower, id=0]Image [name=yellow, id=0]

the first example images objs Id is coming correctly. but after commenting the foreach loop for stream . id is coming as '0' for images obj;

How to have the partial Lowercase Decoding the URL?

I have encoded link. After the decoding

String decodedURL = URLDecoder.decode(encodedLink, "UTF-8");
String delims = "[u, &]";
String[] splittedURL = decodedURL.split(delims);
String resultURL = splittedURL[1].substring(1);

I get next link:

http://example.com?I=vNjWAwAAAABfva85AgAAAABR%2Fv%2F%2F%2FwD%2F%2F%2F%2F%2FBGNhdnMA%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F8%3D

But in this part:

%2Fv%2F%2F%2FwD%2F%2F%2F%2F%2FBGNhdnMA%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F8%3D

I need to have only "F" and "D" letters in Lowercase: F -> f, D -> d. Is it possible to perform decoding after which this letters will be in Lowercase without splitting the decoded link and make necessary letters

.toLowerCase();

Why do I get compilation error when doing flatmap() call?
Let's say I have the following object:
```
public class DaylyData {

private Date date;
private List<Integer> numersList;

// standard getters/setters

public Map<Integer, Date> getIntToDate() {
    Map<Integer, Date> resultMap = new HashMap<>();
    for(Integer number : getNumersList()) {
        resultMap.put(number, getDate());
    }
    return resultMap;
}
```
Now, let say I have a list of DaylyData: List<DaylyData> resultList.

What will be the result of the following:
```
resultList.stream().flatMap(entity -> entity.getIntToDate());
```
If I assign the result of that to Stream<Map<Integer, Date>>, I am getting Type mismatch: cannot convert from Map<Integer,Date> to Stream<? extends Map<Integer,Date>>.

Thanks in advance.

Map<String, Set<T> > to List in Java 8

So I have this class :

 class User {
    public String name;
    public Integer age;

    public User(String name, Integer age) {
        this.name = name;
        this.age = age;
    }
}

And then this Map:

 Map<String, Set<Integer>> map = new HashMap<>();
 map.put("User", Set.of(18, 19, 20, 21));

I want to obtain a list of 4 users here:

name : "John" , age : 18
name : "John" , age : 19
....
....

Any ideas how can I do it with streams in Java 8 ?

Related questions

Restrict CPU usage by java8 parallel stream/High CPU usage by Java 8 parallel stream

2 answers

See also questions close to this topic