If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Database Server Software > DB2 > active data - bufferpool sizing

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 05-01-11, 19:47
db2girl db2girl is offline
∞∞∞∞∞∞
 
Join Date: Aug 2008
Location: Toronto, Canada
Posts: 1,816
active data - bufferpool sizing

I have a question related to page #7 of the following presentation:
http://www.ne-imug.com/Presentations...20Features.pdf


"Advisable to allocate 5% of active data for bufferpool sizing"


I understand about active data, but why 5% for bufferpools. What does it mean?
Reply With Quote
  #2 (permalink)  
Old 05-01-11, 21:36
n_i n_i is offline
:-)
 
Join Date: Jun 2003
Location: Toronto, Canada
Posts: 4,449
I think he means that bufferpools should be sized at 5% of the data volume (on disk). In other words, if your database is 100 GB, you should try to allocate 5 GB for bufferpools.
Reply With Quote
  #3 (permalink)  
Old 05-01-11, 22:38
db2girl db2girl is offline
∞∞∞∞∞∞
 
Join Date: Aug 2008
Location: Toronto, Canada
Posts: 1,816
I guess my question is why only 5% of active data (we need to exclude dormant data). Is 5% sufficient? Do they consider just raw data or include indexes, temps...
Reply With Quote
  #4 (permalink)  
Old 05-01-11, 23:39
Marcus_A Marcus_A is offline
Registered User
 
Join Date: May 2003
Location: USA
Posts: 5,198
I don't know what it means, but I allocate at least 50% of physical memory for bufferpools (assuming it does not exceed the size of the database). Obviously, if multiple databases are on the same server, then that needs to be taken into account.

In looking at the entire presentation, it looks like the presenter is mixing DPF recommendations with general recommendations, without making it clear which is which. Maybe that is the source of the confusion.

Regarding the rest of the presentation, Currently Committed (CUR_COMMIT db cfg) is fools gold. If this is implemented, programmers will very likely start writing really bad code (poor concurrency, lack of commits, failure to close transactions) without getting any deadlocks, and DB2 will start having to go deeper and deeper into the logs (including to logs on disk) to retrieve uncommitted data, and performance will eventually deteriorate to unacceptable level.
__________________
M. A. Feldman
IBM Certified DBA on DB2 for Linux, UNIX, and Windows
IBM Certified DBA on DB2 for z/OS and OS/390
Reply With Quote
  #5 (permalink)  
Old 05-02-11, 06:52
dr_te_z dr_te_z is offline
Registered User
 
Join Date: Jan 2009
Location: Zoetermeer, Holland
Posts: 555
Quote:
Originally Posted by Marcus_A View Post
programmers will very likely start writing really bad code
Let me guess: you are married to one
Reply With Quote
  #6 (permalink)  
Old 05-02-11, 10:22
DebianDog DebianDog is offline
Registered User
 
Join Date: Apr 2011
Posts: 17
See what your hit rates are too

Quote:
db2pd -db DBNAME -bufferpools
Reply With Quote
  #7 (permalink)  
Old 05-04-11, 00:02
db2girl db2girl is offline
∞∞∞∞∞∞
 
Join Date: Aug 2008
Location: Toronto, Canada
Posts: 1,816
Quote:
Originally Posted by Marcus_A View Post
I allocate at least 50% of physical memory for bufferpools (assuming it does not exceed the size of the database).

In your production DPF env, what is the size of your bufferpools relative to the db size (across all partitions)?

Let's say, DPF with 2 physical data servers (I excluded admin server) - each data server has 32 GB. I assume all data stored in the db is active. If I allocate 50% of 32 GB for bufferpools on each physical server and apply "Advisable to allocate 5% of active data for bufferpool sizing" recommendation, that would make my db size ~640 GB. Is your db size close to this number?

Last edited by db2girl; 05-04-11 at 05:28.
Reply With Quote
  #8 (permalink)  
Old 05-04-11, 07:56
n_i n_i is offline
:-)
 
Join Date: Jun 2003
Location: Toronto, Canada
Posts: 4,449
Quote:
Originally Posted by BELLO4KA View Post
that would make my db size ~640 GB
At least it's close to the BCU recommendation of 300 GB of data per node.
Reply With Quote
  #9 (permalink)  
Old 05-04-11, 08:53
Marcus_A Marcus_A is offline
Registered User
 
Join Date: May 2003
Location: USA
Posts: 5,198
Quote:
Originally Posted by BELLO4KA View Post
In your production DPF env, what is the size of your bufferpools relative to the db size (across all partitions)?

Let's say, DPF with 2 physical data servers (I excluded admin server) - each data server has 32 GB. I assume all data stored in the db is active. If I allocate 50% of 32 GB for bufferpools on each physical server and apply "Advisable to allocate 5% of active data for bufferpool sizing" recommendation, that would make my db size ~640 GB. Is your db size close to this number?
Yes, it is close for our DPF system. Keep in mind that each of our 2 physical data servers with 32 GB of memory has 4 DPF parititions (8 total for the 2 data servers), each with its own bufferpools. So 5% is probably reasonable for data warehouse, although I have multiple bufferpools, and the dimension tables (small reference tables) are fully covered by bufferpool memory in a different bufferpool than the large fact tables.

For a OLTP application, I would have multiple bufferpools, and different tables would have different percentages of bufferpool behind them:

For example (crude example):
  1. Small tables and small/medium size indexes: Bufferpools 100% or more the size of the data
  2. Medium Size tables and Large indexes: Bufferpools 50% the size of the data
  3. Large tables: Bufferpools 5-20% the size of the data

The above scenario (based on queueing theory) is one reason why STMM control of bufferpools does not work well.
__________________
M. A. Feldman
IBM Certified DBA on DB2 for Linux, UNIX, and Windows
IBM Certified DBA on DB2 for z/OS and OS/390
Reply With Quote
  #10 (permalink)  
Old 05-04-11, 19:00
db2girl db2girl is offline
∞∞∞∞∞∞
 
Join Date: Aug 2008
Location: Toronto, Canada
Posts: 1,816
Quote:
Originally Posted by n_i View Post
At least it's close to the BCU recommendation of 300 GB of data per node.
This is per logical partition, right?
Reply With Quote
  #11 (permalink)  
Old 05-04-11, 19:07
db2girl db2girl is offline
∞∞∞∞∞∞
 
Join Date: Aug 2008
Location: Toronto, Canada
Posts: 1,816
Quote:
Originally Posted by Marcus_A View Post
Yes, it is close for our DPF system. Keep in mind that each of our 2 physical data servers with 32 GB of memory has 4 DPF parititions (8 total for the 2 data servers), each with its own bufferpools. So 5% is probably reasonable for data warehouse, although I have multiple bufferpools, and the dimension tables (small reference tables) are fully covered by bufferpool memory in a different bufferpool than the large fact tables.

For a OLTP application, I would have multiple bufferpools, and different tables would have different percentages of bufferpool behind them:

For example (crude example):
  1. Small tables and small/medium size indexes: Bufferpools 100% or more the size of the data
  2. Medium Size tables and Large indexes: Bufferpools 50% the size of the data
  3. Large tables: Bufferpools 5-20% the size of the data

The above scenario (based on queueing theory) is one reason why STMM control of bufferpools does not work well.

Yes, I omitted the fact that each physical data server has 4 logical partitions (I was using your DPF setup as an example).

Using BCU recommendation of 300 GB of data per logical partition and recommendation to allocate 5% of data for bufferpools, BPs would need 60 GB of RAM per physical data server. But I think BCU documentation mentions a total of 32 GB of RAM per server.


Thanks everyone for the explanation.
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On