A stateful session bean can keep data between client calls. This means resources (memory, db connection, etc.) are held by the bean waiting for another call from the client.
By storing this state on the client and sending it with each request, you can have the same state, but with better scalability.
Stateless services make scaling out easier because it means that once a request has been processed, the next request from that web visitor does not necessarily have to come back to the same machine, which means you can (in theory) have as many machines as you want service the same kind of requests. If your traffic doubles in volume you just add the same number of machines and boom, you can now service twice as many requests.