A 32 bit JVM process can have a theoretical max heap size of 4 GB based on simple unsigned arithmetic. However, it varies according to OS, and most OS limit it to 2-3 GB. [1]
The options -Xms, -Xmx configure the available Java Heap memory which is a portion of the total memory allocated for the JVM process by the underlying OS.
According to the HotSpot JVM spec, a Java thread maintains a single stack for the native code as well as Java code [2] and therefore such a thread actually is a native OS thread with its own stack. The -Xss option can be used to configure the native stack size. The point is: the memory required for Java threads is deducted from the JVM process’ memory but not from the Java Heap. In other words, if you see a “java.lang.OutOfMemoryError :unable to create new native thread” , consider adjusting your Xmx/ms( reduce it ) and Xss (reduce it as well, if the change in Xmx/ms does not work) according to the following equation -
JVM memory = Xmx+(nos of threads)*Xss + any other memory requirement( e.g. PermGen space, etc.)
Note that default value of Xss depends on the OS , typically 256k-512k [3].