Environment : DB2 V9.7 FP3 on Linux

DPF environment . 17 logical nodes spanned across 3 physical nodes


DBM cfg for FCM buffers and channels are set as below on all 17 partitions.

FCM_NUM_BUFFERS(4KB)=200000
FCM_NUM_CHANNELS=8192

I understand FCM is used for communication between logical partitions.

Based on the above parameter values how much amount of FCM memory is allocated per each partition ?

Try db2pd -fcm

This gives the actual number of buffers sent and received. But I wanted to know theorotically what does it mean by setting the value of FCM_NUM_BUFFERS and FCM_NUM_CHANNELS ? How much memory is allocated per partition for FCM based on these values ?

In our environment FCM_NUM_BUFFERS is set to a value of 200,000 4KB pages this should be equal to 762 MB per partition. Totally we have 17 partitions spanned across 3 physical nodes so the maximum FCM memory should come around 17*0.762 GB = 12.96 GB. When I see memory usage in db2top, it gives that the total FCM memory usage as 58 GB. So where is this extra memory coming in to picture? Does FCM_NUM_CHANNELS come in to picture while calculating the memory ?

For FCM_NUM_BUFFERS:

FCM buffers are shared by all logical partitions on the server. For each physical server, FCM_NUM_BUFFERS x (# of logical partitions + 2 failover partitions)
(2 failover partitions is by default, controlled with DB2_NUM_FAILOVER_NODES)

For Linux, db2 calculates a value based on the above + 25% for larger configuration. It's rounded up to multiple of 2GB and then we add on an additional 2GB
(not exactly sure if this is only for automatic setting, I think this is for automatic and fixed value but don't quote me here).


You have 3 physical / 17 logical partitions. I assume one partition is on its own server (for coordinator node), leaving 2 servers with 8 logical partitons each.

For each server with 8 logical partitons:
200000 * (8 + 2) = 2000000 4K = ~7.6GB

If 25% plus rounding... is done for non-automatic setting, then:
7.6GB x 1.25 = 9.5GB, round up to 10GB + 2GB = 12GB


For server with 1 logical partition
200000 * (1 + 2) = 600000 4K = ~2.3GB
... = 6GB


db2 will allocate a fixed size segment based on the above (one segment per server). You can see the size of this segment in the ipcs output - this is allocated but not committed in RAM. You can check the segment size using:

- Find the Id value (3rd column) for FCM in db2pd -memsets
- ipcs | grep for that ID value. The bytes column is the segment size


I think db2top is doing double counting for FCM memory (comes from global snapshot?), should only count once per server. Try using db2pd.


Please open a pmr if you require more info.

Thanks for your great explanation Bella.

I some difference in values which I noticed on the server using db2pd and ipcs

db2 get dbm cfg | grep -i fcm
No. of int. communication buffers(4KB)(FCM_NUM_BUFFERS) = AUTOMATIC(200000)
No. of int. communication channels (FCM_NUM_CHANNELS) = AUTOMATIC(8192)


Admin node: [Only one partition]

db2pd -memsets | head -5; db2pd -memsets | grep -i fcm
Database Partition 0 -- Active -- Up 5 days 05:44:58 -- Date 11/14/2011 00:16:36

Memory Sets:
Name Address Id Size(Kb) Key DBP Type Unrsv(Kb) Used(Kb) HWM(Kb) Cmt(Kb) Uncmt(Kb)
FCM 0x0000000220000000 1760624657 3314112 0xE164E962 0 11 2784704 529408 883328 529408 2784704

ipcs | head -3; ipcs | grep 1760624657

------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0xe164e962 1760624657 lrmp 701 6442450944 5

If you notice, size on the operating system level is 6442450944 bytes which is 6 GB. This is exactly as calculated by you. But why is the size in db2pd –memsets showing 3314112 KB ? This is just 3.1 GB where is the rest of the memory going ?


Data Node1 : [Eight logical partitions]
db2pd -memsets | head -5; db2pd -memsets | grep -i fcm

Database Partition 1 -- Active -- Up 5 days 05:48:14 -- Date 11/14/2011 00:19:54

Memory Sets:
Name Address Id Size(Kb) Key DBP Type Unrsv(Kb) Used(Kb) HWM(Kb) Cmt(Kb) Uncmt(Kb)
FCM 0x0000000220000000 344195095 11130240 0xE164E962 0 11 7284352 3845888 6682624 3845888 7284352

ipcs | head -3; ipcs | grep 344195095
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0xe164e962 344195095 lrmp 701 15032385536 40

Here size on the operating system level is 15032385536 bytes which comes to 14 GB. As per your prediction, it should have been 12 GB. Not sure why it is different here.

Even here db2pd –memsets is showing a size of 11130240 kb, this would come around 10.61 GB. Not sure where the rest of 3.4 GB is being used up ?


Data Node 2: [Eight logical partitions]

db2pd -memsets | head -5; db2pd -memsets | grep -i fcm

Database Partition 9 -- Active -- Up 5 days 05:52:12 -- Date 11/14/2011 00:23:53

Memory Sets:
Name Address Id Size(Kb) Key DBP Type Unrsv(Kb) Used(Kb) HWM(Kb) Cmt(Kb) Uncmt(Kb)
FCM 0x0000000220000000 338296855 11130240 0xE164E962 0 11 7284352 3845888 6682624 3845888 7284352


ipcs | head -3; ipcs | grep 3382968

key shmid owner perms bytes nattch status
0xe164e962 338296855 lrmp 701 15032385536 40

This is similar to that of Data Node 1 where OS size is 15032385536 [14 GB] and db2pd –memsets showing a value of 11130240 kb [10.61 GB]. Not sure where the rest of the memory is used up? And also not sure why it is 14 GB at OS level unlike your calculation which should be 12 GB.

My calculations were based on FCM_NUM_BUFFERS value. There are also channels... The pre-allocated segment size is very close to what I estimated. Note that this is a pre-allocated size (fixed size - can't grow beyond this value), memory in not actually being used/committed unless it's needed. Cmt from memsets is what you need to look at to see the actual memory usage (only count once per physical server).

I see you opened a pmr this morning. You can tell them I'm familiar with this and to engage me, if needed.

Okay, updating the information in PMR requesting it to engage you.

It can't be assigned directly to me, but they will know what to do if needed.