Again i got one issue with servicemix Poller components.
We have two FilePoller components here and we kept the period to 10 minutes. Which means the instance of FilePoller will run at every 10 minutes. Now say if I have two instances, each run at 10 minutes, and 5 minutes gap between them. So it means after every five minutes, files will be picked by one of them. And the WorkManager, which is given to FilePoller as property, has threadPoolSize of 30.
All was working fine with this configuration.30 threads in pool, every five minutes any of two pullers will pick files. But now we have to change it. When we changed the period from 10 minutes to 1 minute, we got strange errors. Yes even you won't believe it. While testing concurrency we got, FileNotFound exception even though the file exists there, and many other concurrency issues. We looked into the logs and found that sometimes the load goes only on one server and the other is always idle doing nothing. Because when it comes, the first one is doing task.
Then we decided to decrease the thredPoolSize. Becasue if puller get 30 files for 30 threads from the pool, and when it comes to 31st file from the folder and if any of previous 30 threads is in the pool after it has completed its job, then it will take the file. It means, though we have pool size of 30, it may be possible that it can process more than 30 files per instance (say 32 to 38 files). So we decided to make pool size to 10.
Now after making 10 it seems to be working fine. Each one is picking almost equal numbers of files and sharing load as it should. Let’s see if more issues are coming or not.
So, key thing to be remembered is
We have two FilePoller components here and we kept the period to 10 minutes. Which means the instance of FilePoller will run at every 10 minutes. Now say if I have two instances, each run at 10 minutes, and 5 minutes gap between them. So it means after every five minutes, files will be picked by one of them. And the WorkManager, which is given to FilePoller as property, has threadPoolSize of 30.
All was working fine with this configuration.30 threads in pool, every five minutes any of two pullers will pick files. But now we have to change it. When we changed the period from 10 minutes to 1 minute, we got strange errors. Yes even you won't believe it. While testing concurrency we got, FileNotFound exception even though the file exists there, and many other concurrency issues. We looked into the logs and found that sometimes the load goes only on one server and the other is always idle doing nothing. Because when it comes, the first one is doing task.
Then we decided to decrease the thredPoolSize. Becasue if puller get 30 files for 30 threads from the pool, and when it comes to 31st file from the folder and if any of previous 30 threads is in the pool after it has completed its job, then it will take the file. It means, though we have pool size of 30, it may be possible that it can process more than 30 files per instance (say 32 to 38 files). So we decided to make pool size to 10.
Now after making 10 it seems to be working fine. Each one is picking almost equal numbers of files and sharing load as it should. Let’s see if more issues are coming or not.
So, key thing to be remembered is
"Keep you threadPoolSize and period on which instance will run in proportion to each other. If period is long then threads can be more but if you want to reduce the period, you must take care of pool size."
0 Comments:
Post a Comment